-- Logs begin at Tue 2022-09-06 03:44:02 UTC, end at Tue 2022-09-06 05:02:33 UTC. -- Sep 06 04:15:58.720510 dv-cinder-856002-1-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Sep 06 04:15:58.727982 dv-cinder-856002-1-iscsi devstack@keystone.service[81687]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Sep 06 04:15:58.733398 dv-cinder-856002-1-iscsi devstack@keystone.service[81687]: *** Starting uWSGI 2.0.18-debian (64bit) on [Tue Sep 6 04:15:58 2022] *** Sep 06 04:15:58.733398 dv-cinder-856002-1-iscsi devstack@keystone.service[81687]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Sep 06 04:15:58.733398 dv-cinder-856002-1-iscsi devstack@keystone.service[81687]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Sep 06 04:15:58.733398 dv-cinder-856002-1-iscsi devstack@keystone.service[81687]: nodename: dv-cinder-856002-1-iscsi Sep 06 04:15:58.733398 dv-cinder-856002-1-iscsi devstack@keystone.service[81687]: machine: x86_64 Sep 06 04:15:58.733398 dv-cinder-856002-1-iscsi devstack@keystone.service[81687]: clock source: unix Sep 06 04:15:58.733398 dv-cinder-856002-1-iscsi devstack@keystone.service[81687]: pcre jit disabled Sep 06 04:15:58.733398 dv-cinder-856002-1-iscsi devstack@keystone.service[81687]: detected number of CPU cores: 4 Sep 06 04:15:58.733398 dv-cinder-856002-1-iscsi devstack@keystone.service[81687]: current working directory: / Sep 06 04:15:58.733398 dv-cinder-856002-1-iscsi devstack@keystone.service[81687]: detected binary path: /usr/bin/uwsgi-core Sep 06 04:15:58.733398 dv-cinder-856002-1-iscsi devstack@keystone.service[81687]: your processes number limit is 47680 Sep 06 04:15:58.733398 dv-cinder-856002-1-iscsi devstack@keystone.service[81687]: your memory page size is 4096 bytes Sep 06 04:15:58.733398 dv-cinder-856002-1-iscsi devstack@keystone.service[81687]: detected max file descriptor number: 2048 Sep 06 04:15:58.733398 dv-cinder-856002-1-iscsi devstack@keystone.service[81687]: lock engine: pthread robust mutexes Sep 06 04:15:58.733398 dv-cinder-856002-1-iscsi devstack@keystone.service[81687]: thunder lock: enabled Sep 06 04:15:58.733398 dv-cinder-856002-1-iscsi devstack@keystone.service[81687]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Sep 06 04:15:58.733398 dv-cinder-856002-1-iscsi devstack@keystone.service[81687]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Sep 06 04:15:58.770855 dv-cinder-856002-1-iscsi devstack@keystone.service[81687]: Python main interpreter initialized at 0x55f2d894f9c0 Sep 06 04:15:58.770855 dv-cinder-856002-1-iscsi devstack@keystone.service[81687]: python threads support enabled Sep 06 04:15:58.770855 dv-cinder-856002-1-iscsi devstack@keystone.service[81687]: your server socket listen backlog is limited to 100 connections Sep 06 04:15:58.770855 dv-cinder-856002-1-iscsi devstack@keystone.service[81687]: your mercy for graceful operations on workers is 90 seconds Sep 06 04:15:58.771204 dv-cinder-856002-1-iscsi devstack@keystone.service[81687]: mapped 403077 bytes (393 KB) for 2 cores Sep 06 04:15:58.771280 dv-cinder-856002-1-iscsi devstack@keystone.service[81687]: *** Operational MODE: preforking *** Sep 06 04:15:58.771549 dv-cinder-856002-1-iscsi devstack@keystone.service[81687]: *** uWSGI is running in multiple interpreter mode *** Sep 06 04:15:58.771549 dv-cinder-856002-1-iscsi devstack@keystone.service[81687]: spawned uWSGI master process (pid: 81687) Sep 06 04:15:58.771750 dv-cinder-856002-1-iscsi systemd[1]: Started Devstack devstack@keystone.service. Sep 06 04:15:58.772329 dv-cinder-856002-1-iscsi devstack@keystone.service[81687]: spawned uWSGI worker 1 (pid: 81697, cores: 1) Sep 06 04:15:58.772658 dv-cinder-856002-1-iscsi devstack@keystone.service[81687]: spawned uWSGI worker 2 (pid: 81698, cores: 1) Sep 06 04:15:58.772815 dv-cinder-856002-1-iscsi devstack@keystone.service[81687]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Sep 06 04:16:03.177340 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 06 04:16:03.177340 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: from cryptography.utils import int_from_bytes Sep 06 04:16:03.178150 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 06 04:16:03.178150 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: from cryptography.utils import int_from_bytes Sep 06 04:16:03.204099 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 06 04:16:03.204099 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: from cryptography.utils import int_from_bytes Sep 06 04:16:03.204601 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 06 04:16:03.204601 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: from cryptography.utils import int_from_bytes Sep 06 04:16:03.235917 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Sep 06 04:16:03.236114 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Sep 06 04:16:03.236279 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] command line args: [] {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Sep 06 04:16:03.236551 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Sep 06 04:16:03.236732 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] ================================================================================ {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Sep 06 04:16:03.236994 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] admin_token = **** {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 04:16:03.237242 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] config_dir = [] {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 04:16:03.237507 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 04:16:03.237647 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] config_source = [] {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 04:16:03.237883 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] debug = True {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 04:16:03.238037 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 04:16:03.239123 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 04:16:03.239123 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 04:16:03.239123 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 04:16:03.239123 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 04:16:03.239373 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] list_limit = None {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 04:16:03.239501 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] log_config_append = None {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 04:16:03.239634 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 04:16:03.239815 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] log_dir = None {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 04:16:03.239999 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] log_file = None {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 04:16:03.240681 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 04:16:03.240681 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 04:16:03.240851 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 04:16:03.241202 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 04:16:03.241202 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 04:16:03.241342 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 04:16:03.241553 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 04:16:03.241803 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 04:16:03.242094 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 04:16:03.243467 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 04:16:03.243714 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 04:16:03.243956 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 04:16:03.244212 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 04:16:03.244465 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 04:16:03.244746 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 04:16:03.245249 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.30/identity {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 04:16:03.254266 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] publish_errors = False {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 04:16:03.254266 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 04:16:03.254266 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 04:16:03.254266 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 04:16:03.254266 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 04:16:03.254266 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 04:16:03.254266 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] standard_threads = False {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 04:16:03.254266 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 04:16:03.254266 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 04:16:03.254789 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 04:16:03.254789 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] use_journal = False {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 04:16:03.254789 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] use_json = False {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 04:16:03.254789 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] use_stderr = False {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 04:16:03.254789 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] use_syslog = False {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 04:16:03.254789 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 04:16:03.254789 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.254789 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.254789 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.255508 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.255508 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.255508 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.255508 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.255508 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.255508 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.255508 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.255508 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.255508 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.255948 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.255948 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.255948 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.255948 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.255948 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.255948 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.255948 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.255948 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.255948 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.256383 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.256383 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.256383 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.256383 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.256383 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.256383 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.256383 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.256383 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.256383 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.256877 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.256877 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.256877 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.256877 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.256877 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.257154 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.257154 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.257423 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.257860 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.258513 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.258758 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.258998 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.259236 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.259471 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.259714 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.259966 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.260203 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.260450 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.260760 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.260919 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.261159 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.261436 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.261673 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.261916 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.262157 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.262398 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.262638 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.262878 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.263113 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.263367 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.263607 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.263846 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.264081 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.264342 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.264581 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.264816 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.265058 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.265297 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.265593 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.265837 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.266088 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.266337 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.266576 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.266815 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.267052 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.267302 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.267540 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.267780 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.268026 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.268271 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.268521 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.268760 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.268996 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.269234 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.269495 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.269733 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.269996 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.270242 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.270444 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Sep 06 04:16:03.270522 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.270642 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Sep 06 04:16:03.270730 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.270837 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] command line args: [] {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Sep 06 04:16:03.270951 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.271108 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Sep 06 04:16:03.271182 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.271288 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] ================================================================================ {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Sep 06 04:16:03.271423 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.271545 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] admin_token = **** {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 04:16:03.271662 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.271794 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] config_dir = [] {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 04:16:03.271930 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.272006 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 04:16:03.272177 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.272306 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] config_source = [] {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 04:16:03.272368 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.272425 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] debug = True {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 04:16:03.272590 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 04:16:03.272711 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.272785 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.272858 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 04:16:03.272946 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 04:16:03.273021 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.273208 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 04:16:03.273408 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.273548 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 04:16:03.273851 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] list_limit = None {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 04:16:03.274036 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] log_config_append = None {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 04:16:03.274116 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.274218 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 04:16:03.274342 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.274424 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] log_dir = None {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 04:16:03.274565 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] log_file = None {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 04:16:03.274638 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.274803 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 04:16:03.274891 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.275053 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 04:16:03.275134 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.275302 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 04:16:03.275376 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.275540 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] auth.external = None {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.275626 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 04:16:03.275721 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 04:16:03.275826 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.275926 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 04:16:03.276065 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.276169 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 04:16:03.276302 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.276419 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 04:16:03.276549 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] auth.password = None {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.276655 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 04:16:03.276761 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] auth.token = None {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.276856 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 04:16:03.276989 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.277099 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 04:16:03.277232 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.277327 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 04:16:03.277504 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 04:16:03.277588 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.277754 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 04:16:03.277831 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.278031 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 04:16:03.278113 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.278293 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.278524 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.30/identity {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 04:16:03.278632 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.278704 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] publish_errors = False {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 04:16:03.278769 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] credential.caching = True {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.278927 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 04:16:03.279003 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.279160 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 04:16:03.279241 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.279361 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 04:16:03.279474 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.279591 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 04:16:03.279722 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.279807 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 04:16:03.279971 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.280047 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] standard_threads = False {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 04:16:03.280211 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.280289 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 04:16:03.280461 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.280548 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 04:16:03.280661 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 04:16:03.280733 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.280845 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] use_journal = False {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 04:16:03.280942 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.281058 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] use_json = False {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 04:16:03.281144 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] use_stderr = False {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 04:16:03.281214 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.281288 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] use_syslog = False {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 04:16:03.281499 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.281586 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 04:16:03.281852 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.281943 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.282049 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.282124 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.282236 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.282311 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.282475 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.282559 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.282718 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] federation.caching = True {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.282793 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.282954 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.283029 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.283200 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.283287 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.283469 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.283578 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.283652 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.283720 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.283903 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.283982 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.284142 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.284219 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.284380 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.284461 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.284606 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.284689 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.284855 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.284930 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.285092 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.285167 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.285358 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.285764 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.285764 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] identity.caching = True {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.285992 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.286066 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.286135 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.286204 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.286272 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.286348 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.286419 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.286521 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.286658 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.286773 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.286916 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.286993 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.287182 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.287264 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.287440 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.287515 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.287693 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.287770 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.287928 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.288011 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.288162 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.288238 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.288394 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.288470 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.288627 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.288701 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.289614 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.289745 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.289831 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.289957 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.290141 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.290224 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.290392 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.290468 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.290643 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.290720 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.290882 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.290957 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.291132 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.291209 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.291368 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.291445 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.291606 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.291683 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.291844 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.291920 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.292077 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.292157 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.292306 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.292392 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.292545 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.292629 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.292778 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.292855 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.293016 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.293100 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.293247 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.293344 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.293512 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.293598 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.293753 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.293833 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.293986 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.294062 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.294223 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.294301 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.294477 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.294564 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.294712 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.294809 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.294954 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.295031 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.295189 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.295267 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.295432 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.295511 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.295674 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.295750 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.295921 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.296005 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.296163 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.296242 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.296394 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.296469 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.296642 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.296718 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.296875 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.297018 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.297105 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.297258 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.297380 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.297523 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.297610 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.297780 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.297857 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.298035 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.298118 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.298272 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.298347 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.298510 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.298585 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.298756 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.298903 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.298977 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] ldap.user = None {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.299047 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.299220 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.299302 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.299466 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.299550 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.299704 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.299779 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.299937 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.300012 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.300175 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.300259 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.300424 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.300500 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.300658 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.300797 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.300894 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.301014 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.301118 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.301196 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.301381 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.301461 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.301624 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.302053 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.302143 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.302222 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.302222 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.302349 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.302418 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.302495 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.302566 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.302635 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.302765 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.302842 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.303027 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.303112 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.303257 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.303332 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.303510 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.303601 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.303742 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.303827 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.303977 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.304053 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.304211 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.304295 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.304446 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.304557 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.305469 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.305728 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.305804 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.305971 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.306051 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.306231 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.306312 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.306484 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.306560 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.306720 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.306803 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.306954 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.307030 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.307197 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.307273 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.307436 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.307520 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.307675 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.307750 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.307920 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.307996 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.308161 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.308244 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.308398 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.308475 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.308639 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.308717 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.308885 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] auth.external = None {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.308968 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] resource.caching = True {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.309121 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.309197 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.309398 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.309477 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.309644 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.309733 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.309887 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] auth.password = None {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.309963 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.310125 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] auth.token = None {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.310201 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.310358 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.310441 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.310596 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.310671 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.310834 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.310909 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.311071 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.311155 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.311306 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] role.caching = True {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.311387 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.311557 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] role.driver = sql {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.311634 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.311795 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.311958 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.312030 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] credential.caching = True {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.312091 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.312274 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.312358 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.312520 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.312608 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.312766 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.312849 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.313012 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.313098 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.313257 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.317398 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.317476 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.317536 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.317608 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.317668 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.317668 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.317808 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.317881 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.317951 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.318021 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.318091 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.318160 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.318238 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.318309 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.318379 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.318379 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.318504 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.318581 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.318653 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.318722 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.318792 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.318862 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] federation.caching = True {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.318930 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.319007 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.319078 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.319148 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.319216 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.319286 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.319355 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.319432 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.319504 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.319574 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.319643 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.319713 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.319781 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.319857 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.319929 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.319998 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.320067 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.320144 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.320217 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.320294 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.320365 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.320435 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.320504 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.320573 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] identity.caching = True {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.320638 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.320705 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.320765 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.320822 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.320887 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.320944 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.321021 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.321103 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.321175 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] token.caching = True {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.321243 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.321333 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.321398 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.321467 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.321549 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.321549 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.321549 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.321733 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.321803 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.321872 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.321949 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.322021 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.322090 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.322159 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.322227 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.322296 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.322373 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.322444 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.322513 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.322581 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.322690 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.322770 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.322916 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.322991 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.323061 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.323130 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.323206 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.323277 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.323354 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.323483 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.323661 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.323751 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.323834 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.323953 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.324062 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.324224 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.324303 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.324490 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] database.connection = **** {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.324567 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.324737 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.324815 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.324987 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.325064 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.325222 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.325355 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.325489 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.325565 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.325730 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.325809 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.325964 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.326039 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.326200 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.326276 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.326440 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.326519 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.326673 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.326748 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.326914 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.326990 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.327144 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.327221 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.327384 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.327460 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.327621 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.327696 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.327862 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.327939 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.328098 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.328178 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.328328 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.328404 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.328570 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.328648 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.328811 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.328895 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.328987 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81698) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Sep 06 04:16:03.329153 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.329414 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.329670 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.329921 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.330161 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.330401 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.330639 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.330874 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] ldap.user = None {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.331116 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.331377 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.331612 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.331861 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.332111 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.332366 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.332602 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.332835 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.333072 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.333321 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.333563 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.333796 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.334045 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.334295 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.334542 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.334828 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.335080 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.335313 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.335564 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.335827 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.336066 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.336308 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.336542 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.336788 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.337144 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.337429 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.337693 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.337929 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.338177 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.338418 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.338656 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.338898 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.339148 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.339392 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.339628 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.339869 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.340114 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] resource.caching = True {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.340368 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.340620 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.340862 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.341115 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.341395 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.341636 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.341887 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.342133 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.342376 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.342614 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] role.caching = True {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.342863 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] role.driver = sql {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.343098 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.343350 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.343601 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.343853 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.344104 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.344352 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.344600 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.344849 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.345098 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.345353 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.345606 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.345855 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.346104 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.346354 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.346759 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.346998 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.347250 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.347503 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.347755 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.348008 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.348329 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.348708 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.349007 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.349283 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.349576 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.349838 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.350096 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.350371 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.350652 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.350922 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.351177 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.351933 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.352285 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.352542 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] token.caching = True {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.352795 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.353059 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.353327 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.353604 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.353866 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.354114 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.354368 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.354621 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.354879 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.355126 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.355373 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.355617 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.355871 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.356159 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.356400 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.356650 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.356926 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.357198 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] database.connection = **** {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.357492 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.357753 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.358002 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.358257 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.358502 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.358747 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.359050 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.359236 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.359491 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.359611 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81698) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 04:16:03.359737 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.359846 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81698) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 04:16:03.359989 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.360076 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81698) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 04:16:03.360233 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.360491 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.360736 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.360989 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.361227 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.361508 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.361740 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 04:16:03.361919 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Sep 06 04:16:03.369636 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81698) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 04:16:03.388449 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81697) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 04:16:03.388688 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81697) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 04:16:03.388918 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81697) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 04:16:03.397457 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81697) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 04:16:03.399202 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81698) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 04:16:03.399438 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81698) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 04:16:03.416606 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81697) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 04:16:03.416845 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81697) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 04:16:03.462210 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81698) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 04:16:03.462444 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81697) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 04:16:03.517925 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81698) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 04:16:03.518159 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81698) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 04:16:03.518899 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81697) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 04:16:03.519146 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81697) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 04:16:03.684635 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81697) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 04:16:03.684820 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81698) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 04:16:03.684988 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81697) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 04:16:03.685072 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81698) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 04:16:03.685136 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81697) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 04:16:03.685266 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81698) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 04:16:03.685369 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81697) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 04:16:03.685518 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81698) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 04:16:03.685611 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81697) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 04:16:03.685739 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81697) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 04:16:03.685824 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81698) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 04:16:03.685950 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81698) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 04:16:03.708702 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81698) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 04:16:03.708936 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81698) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 04:16:03.709019 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81697) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 04:16:03.709165 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81698) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 04:16:03.709246 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81697) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 04:16:03.709414 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81698) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 04:16:03.709513 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81697) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 04:16:03.709636 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81698) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 04:16:03.709718 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81697) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 04:16:03.709852 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81698) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 04:16:03.709934 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81697) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 04:16:03.710070 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81697) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 04:16:03.710151 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81698) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 04:16:03.710294 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81697) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 04:16:03.721899 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81698) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 04:16:03.721999 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81697) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 04:16:03.734765 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81698) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 04:16:03.735095 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81697) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 04:16:03.735180 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81698) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 04:16:03.735334 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81697) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 04:16:03.735471 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81698) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 04:16:03.735553 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81697) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 04:16:03.735783 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81697) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 04:16:03.735873 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81698) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 04:16:03.736010 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81697) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 04:16:03.736186 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81698) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 04:16:03.736267 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81697) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 04:16:03.736441 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81697) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 04:16:03.736586 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81698) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 04:16:03.736981 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81698) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 04:16:03.754854 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 04:16:03.755076 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 06 04:16:03.755425 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 04:16:03.755633 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 06 04:16:03.755914 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 04:16:03.756186 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 04:16:03.756292 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81698) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 04:16:03.756454 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 04:16:03.756544 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81698) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 06 04:16:03.756721 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 04:16:03.756831 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81698) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 04:16:03.757032 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81698) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 06 04:16:03.757322 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81698) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 04:16:03.757593 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81698) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 04:16:03.758413 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81698) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 04:16:03.758673 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81698) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 04:16:03.767104 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 04:16:03.767393 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 04:16:03.767670 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 04:16:03.767955 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 04:16:03.768231 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81698) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 04:16:03.768510 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81698) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 04:16:03.768778 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81698) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 04:16:03.769059 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81698) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 04:16:03.775210 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81697) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 06 04:16:03.776165 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81698) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 06 04:16:03.778186 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81697) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 06 04:16:03.778479 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 04:16:03.778768 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 04:16:03.779061 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 04:16:03.779143 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81698) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 06 04:16:03.779353 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 04:16:03.779434 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81698) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 04:16:03.779622 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 04:16:03.779703 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81698) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 04:16:03.779899 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81698) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 04:16:03.779981 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 04:16:03.780176 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 04:16:03.780256 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81698) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 04:16:03.780435 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81698) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 04:16:03.780531 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 04:16:03.780702 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81698) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 04:16:03.780782 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 04:16:03.780976 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81698) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 04:16:03.781055 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 04:16:03.781247 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81698) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 04:16:03.781545 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81698) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 04:16:03.781815 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81698) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 04:16:03.803924 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 04:16:03.805243 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81698) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 04:16:03.805961 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81697) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 06 04:16:03.806250 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 04:16:03.806785 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81698) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 06 04:16:03.807071 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81698) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 04:16:03.810846 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81697) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 06 04:16:03.811139 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 04:16:03.811432 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 04:16:03.811750 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81698) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 06 04:16:03.812034 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81698) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 04:16:03.812315 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81698) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 04:16:03.818024 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81697) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 06 04:16:03.818311 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 04:16:03.818657 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81698) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 06 04:16:03.819136 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81698) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 04:16:03.823184 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81697) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 06 04:16:03.823479 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 04:16:03.823768 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 04:16:03.824058 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 04:16:03.824342 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 04:16:03.825099 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81698) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 06 04:16:03.825419 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81698) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 04:16:03.825713 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81698) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 04:16:03.826004 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81698) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 04:16:03.826286 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81698) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 04:16:03.826367 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 04:16:03.826562 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81698) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 04:16:03.826641 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 04:16:03.826829 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81698) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 04:16:03.826909 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 04:16:03.827099 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81698) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 04:16:03.842611 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81698) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 04:16:03.842709 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 04:16:03.842890 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81698) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 04:16:03.842977 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 04:16:03.846760 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81698) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 06 04:16:03.846907 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81697) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 06 04:16:03.850012 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81698) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 04:16:03.850094 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 04:16:03.850309 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81698) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 04:16:03.850392 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 04:16:03.855644 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81697) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 06 04:16:03.856486 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81698) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 06 04:16:03.858854 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81697) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 06 04:16:03.861240 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81698) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 06 04:16:03.862058 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 04:16:03.862377 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 04:16:03.862667 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 04:16:03.862956 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 04:16:03.863248 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 04:16:03.863893 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 04:16:03.865379 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81698) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 04:16:03.865687 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81698) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 04:16:03.865974 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81698) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 04:16:03.866275 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81698) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 04:16:03.866563 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81698) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 04:16:03.866856 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81698) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 04:16:03.884938 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81697) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 06 04:16:03.885089 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81698) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 06 04:16:03.885281 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 04:16:03.885404 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81698) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 04:16:03.885539 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 04:16:03.885650 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81698) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 04:16:03.885810 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 04:16:03.885907 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81698) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 04:16:03.892654 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 04:16:03.892739 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81698) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 04:16:03.894295 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81698) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 04:16:03.894380 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 04:16:03.895812 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81698) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 04:16:03.895925 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 04:16:03.896089 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81698) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 04:16:03.896209 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 04:16:03.899403 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81698) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 06 04:16:03.899500 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81697) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 06 04:16:03.899694 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81698) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 04:16:03.899778 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 04:16:03.899977 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81698) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 04:16:03.900059 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 04:16:03.900256 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81698) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 04:16:03.900347 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 04:16:03.900539 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 04:16:03.900646 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81698) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 04:16:03.911774 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81698) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 06 04:16:03.911962 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81697) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 06 04:16:03.912058 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81698) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 04:16:03.912256 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 04:16:03.912338 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81698) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 04:16:03.912544 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 04:16:03.912626 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81698) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 04:16:03.912833 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 04:16:03.912955 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81698) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 04:16:03.913121 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 04:16:03.913236 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81698) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 04:16:03.913431 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 04:16:03.913527 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81698) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 04:16:03.913729 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 04:16:03.928918 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81698) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 06 04:16:03.929370 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81697) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 06 04:16:03.932397 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81698) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 06 04:16:03.933203 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81697) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 06 04:16:03.935312 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81698) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 04:16:03.936136 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 04:16:03.936775 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81698) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 04:16:03.937599 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 04:16:03.938261 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81698) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 06 04:16:03.938548 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81698) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 04:16:03.938942 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81698) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 04:16:03.939088 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81697) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 06 04:16:03.939378 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 04:16:03.939690 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 04:16:03.945294 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81698) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 04:16:03.946441 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 04:16:03.946779 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81698) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 06 04:16:03.947910 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81697) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 06 04:16:03.949688 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81698) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 04:16:03.950085 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81698) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 04:16:03.950372 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81698) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 04:16:03.950649 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81698) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 04:16:03.950876 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 04:16:03.951179 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 04:16:03.951472 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 04:16:03.951753 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 04:16:03.958472 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81698) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 06 04:16:03.958787 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81698) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 04:16:03.959078 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81698) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 04:16:03.959787 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81697) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 06 04:16:03.960086 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 04:16:03.960383 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 04:16:03.966040 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81698) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 06 04:16:03.966327 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81698) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 04:16:03.966611 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81698) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 04:16:03.966890 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81698) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 04:16:03.967174 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81698) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 04:16:03.967417 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81697) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 06 04:16:03.967498 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81698) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 04:16:03.967711 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 04:16:03.967799 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81698) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 04:16:03.967976 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81698) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 04:16:03.968056 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 04:16:03.968243 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81698) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 04:16:03.968322 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 04:16:03.968527 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81698) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 04:16:03.968615 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 04:16:03.968806 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81698) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 04:16:03.968885 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 04:16:03.969080 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81698) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 04:16:03.969159 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 04:16:03.969367 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81698) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 04:16:03.969451 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 04:16:03.969636 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81698) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 04:16:03.969717 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 04:16:03.969925 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 04:16:03.970213 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 04:16:03.970482 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 04:16:03.970753 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 04:16:03.971030 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 04:16:04.004599 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x55f2d894f9c0 pid: 81698 (default app) Sep 06 04:16:04.006138 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x55f2d894f9c0 pid: 81697 (default app) Sep 06 04:16:04.008378 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b778c701-d9a8-4b39-af6a-40282a34b31b None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:16:04.008792 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b778c701-d9a8-4b39-af6a-40282a34b31b None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:16:04.009068 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b778c701-d9a8-4b39-af6a-40282a34b31b None None] PATH_INFO: `/v3` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:16:04.013206 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1/1] 10.222.0.30 () {50 vars in 791 bytes} [Tue Sep 6 04:16:04 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:16:11.556121 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52b6fb1-2629-4495-ab02-5a67296baa19 None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:16:11.556634 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52b6fb1-2629-4495-ab02-5a67296baa19 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:16:11.556717 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52b6fb1-2629-4495-ab02-5a67296baa19 None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:16:11.560992 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1/2] 10.222.0.30 () {54 vars in 927 bytes} [Tue Sep 6 04:16:11 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 06 04:16:11.566603 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd35d69b-2daf-488b-bf4a-b829242dca7c None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:16:11.567053 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd35d69b-2daf-488b-bf4a-b829242dca7c None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:16:11.567150 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd35d69b-2daf-488b-bf4a-b829242dca7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:16:11.638677 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: INFO dbcounter [None req-dd35d69b-2daf-488b-bf4a-b829242dca7c None None] Registered counter for database keystone Sep 06 04:16:11.657831 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG oslo_db.sqlalchemy.engines [None req-dd35d69b-2daf-488b-bf4a-b829242dca7c None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81698) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 06 04:16:11.664120 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writer thread running {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 06 04:16:11.845561 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG passlib.handlers.bcrypt [None req-dd35d69b-2daf-488b-bf4a-b829242dca7c None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81698) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Sep 06 04:16:11.845942 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG passlib.handlers.bcrypt [None req-dd35d69b-2daf-488b-bf4a-b829242dca7c None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81698) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Sep 06 04:16:11.904479 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-dd35d69b-2daf-488b-bf4a-b829242dca7c None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:16:11.906066 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-dd35d69b-2daf-488b-bf4a-b829242dca7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:16:12.034710 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 2/3] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Sep 6 04:16:11 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 469 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Sep 06 04:16:12.515273 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52b6fb1-2629-4495-ab02-5a67296baa19 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:16:12.584400 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: INFO dbcounter [None req-e52b6fb1-2629-4495-ab02-5a67296baa19 None None] Registered counter for database keystone Sep 06 04:16:12.607111 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG oslo_db.sqlalchemy.engines [None req-e52b6fb1-2629-4495-ab02-5a67296baa19 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81697) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 06 04:16:12.613384 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writer thread running {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 06 04:16:12.733389 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52b6fb1-2629-4495-ab02-5a67296baa19 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:16:12.737557 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef1c8a8-5deb-4200-8a84-3aa27a444f04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:16:12.738225 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef1c8a8-5deb-4200-8a84-3aa27a444f04 admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:16:12.738540 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef1c8a8-5deb-4200-8a84-3aa27a444f04 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:16:12.738851 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef1c8a8-5deb-4200-8a84-3aa27a444f04 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:16:12.747286 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef1c8a8-5deb-4200-8a84-3aa27a444f04 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:16:12.800426 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: WARNING keystone.common.rbac_enforcer.enforcer [None req-9ef1c8a8-5deb-4200-8a84-3aa27a444f04 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Sep 06 04:16:12.800772 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef1c8a8-5deb-4200-8a84-3aa27a444f04 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:16:12.810814 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 2/4] 10.222.0.30 () {58 vars in 1324 bytes} [Tue Sep 6 04:16:12 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 06 04:16:12.816854 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd35d69b-2daf-488b-bf4a-b829242dca7c None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:16:12.831912 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd35d69b-2daf-488b-bf4a-b829242dca7c None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:16:12.836375 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af0ea06-3305-4674-b79c-7a45cd6b0b0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:16:12.837522 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af0ea06-3305-4674-b79c-7a45cd6b0b0d admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:16:12.837806 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af0ea06-3305-4674-b79c-7a45cd6b0b0d admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:16:12.838073 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af0ea06-3305-4674-b79c-7a45cd6b0b0d admin admin] PATH_INFO: `/v3/projects/8b3067b612904279b76189015d91b056` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:16:12.839666 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af0ea06-3305-4674-b79c-7a45cd6b0b0d admin admin] RBAC: Authorizing `identity:get_project(project_id=8b3067b612904279b76189015d91b056)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:16:12.893215 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: WARNING keystone.common.rbac_enforcer.enforcer [None req-9af0ea06-3305-4674-b79c-7a45cd6b0b0d admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Sep 06 04:16:12.893572 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af0ea06-3305-4674-b79c-7a45cd6b0b0d admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:16:12.898899 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 3/5] 10.222.0.30 () {56 vars in 1207 bytes} [Tue Sep 6 04:16:12 2022] GET /identity/v3/projects/8b3067b612904279b76189015d91b056 => generated 344 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 04:16:14.767108 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c28beea-c08d-438d-a250-b4850962319d None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:16:14.767606 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c28beea-c08d-438d-a250-b4850962319d None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:16:14.767606 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c28beea-c08d-438d-a250-b4850962319d None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:16:14.771673 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 3/6] 10.222.0.30 () {54 vars in 927 bytes} [Tue Sep 6 04:16:14 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 04:16:14.777104 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e388f1f9-e1b6-43cd-82c5-d487966089a0 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:16:14.777543 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e388f1f9-e1b6-43cd-82c5-d487966089a0 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:16:14.777673 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e388f1f9-e1b6-43cd-82c5-d487966089a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:16:14.797841 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-e388f1f9-e1b6-43cd-82c5-d487966089a0 None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:16:14.799428 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-e388f1f9-e1b6-43cd-82c5-d487966089a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:16:14.807836 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 4/7] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Sep 6 04:16:14 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Sep 06 04:16:15.084967 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c28beea-c08d-438d-a250-b4850962319d None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:16:15.097068 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c28beea-c08d-438d-a250-b4850962319d None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:16:15.101326 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66de612-734f-4c43-91db-27b892534f33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:16:15.101919 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66de612-734f-4c43-91db-27b892534f33 admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:16:15.102198 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66de612-734f-4c43-91db-27b892534f33 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:16:15.102469 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66de612-734f-4c43-91db-27b892534f33 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:16:15.109980 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66de612-734f-4c43-91db-27b892534f33 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:16:15.111750 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66de612-734f-4c43-91db-27b892534f33 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:16:15.121374 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 4/8] 10.222.0.30 () {58 vars in 1324 bytes} [Tue Sep 6 04:16:15 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 06 04:16:15.126594 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e388f1f9-e1b6-43cd-82c5-d487966089a0 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:16:15.137084 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e388f1f9-e1b6-43cd-82c5-d487966089a0 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:16:15.141536 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c16822-3585-40a2-9cc7-94320bd2f029 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:16:15.142163 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c16822-3585-40a2-9cc7-94320bd2f029 admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:16:15.142446 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c16822-3585-40a2-9cc7-94320bd2f029 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:16:15.142722 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c16822-3585-40a2-9cc7-94320bd2f029 admin admin] PATH_INFO: `/v3/users/b7c5062264774b85bf945fb9ac7602db` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:16:15.144039 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c16822-3585-40a2-9cc7-94320bd2f029 admin admin] RBAC: Authorizing `identity:get_user(user_id=b7c5062264774b85bf945fb9ac7602db)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:16:15.145911 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c16822-3585-40a2-9cc7-94320bd2f029 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:16:15.147429 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 5/9] 10.222.0.30 () {56 vars in 1198 bytes} [Tue Sep 6 04:16:15 2022] GET /identity/v3/users/b7c5062264774b85bf945fb9ac7602db => generated 247 bytes in 21 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:16:17.136410 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeab9618-33c1-484d-a649-cf6e83627256 None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:16:17.136924 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeab9618-33c1-484d-a649-cf6e83627256 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:16:17.137023 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeab9618-33c1-484d-a649-cf6e83627256 None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:16:17.137641 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b769756f-2872-42be-9f42-0e65ede8ee27 None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:16:17.138001 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b769756f-2872-42be-9f42-0e65ede8ee27 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:16:17.138189 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b769756f-2872-42be-9f42-0e65ede8ee27 None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:16:17.138302 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 5/10] 10.222.0.30 () {54 vars in 927 bytes} [Tue Sep 6 04:16:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 04:16:17.139644 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 6/11] 10.222.0.30 () {54 vars in 927 bytes} [Tue Sep 6 04:16:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 06 04:16:17.143097 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7a5afb-b897-4972-befb-c2b6f8b2ec86 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:16:17.143380 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7a5afb-b897-4972-befb-c2b6f8b2ec86 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:16:17.143657 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7a5afb-b897-4972-befb-c2b6f8b2ec86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:16:17.144524 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cde67f-a490-419b-8e58-4963b79dbd35 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:16:17.144814 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cde67f-a490-419b-8e58-4963b79dbd35 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:16:17.145090 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cde67f-a490-419b-8e58-4963b79dbd35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:16:17.163997 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-14cde67f-a490-419b-8e58-4963b79dbd35 None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:16:17.165182 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-14cde67f-a490-419b-8e58-4963b79dbd35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:16:17.185973 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG passlib.handlers.bcrypt [None req-9b7a5afb-b897-4972-befb-c2b6f8b2ec86 None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81697) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Sep 06 04:16:17.186367 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG passlib.handlers.bcrypt [None req-9b7a5afb-b897-4972-befb-c2b6f8b2ec86 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81697) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Sep 06 04:16:17.235625 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-9b7a5afb-b897-4972-befb-c2b6f8b2ec86 None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:16:17.237218 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-9b7a5afb-b897-4972-befb-c2b6f8b2ec86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:16:17.243790 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 7/12] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Sep 6 04:16:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 06 04:16:17.348220 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 6/13] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Sep 6 04:16:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 04:16:17.526789 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14cde67f-a490-419b-8e58-4963b79dbd35 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:16:17.551949 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:16:17.619176 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7a5afb-b897-4972-befb-c2b6f8b2ec86 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:16:17.633091 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14cde67f-a490-419b-8e58-4963b79dbd35 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:16:17.673446 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0bdc0cc-2d35-42a3-b0df-c619f0607c33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:16:17.674156 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bdc0cc-2d35-42a3-b0df-c619f0607c33 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:16:17.674443 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bdc0cc-2d35-42a3-b0df-c619f0607c33 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:16:17.674718 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bdc0cc-2d35-42a3-b0df-c619f0607c33 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:16:17.687295 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0bdc0cc-2d35-42a3-b0df-c619f0607c33 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 04:16:17.696308 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-d0bdc0cc-2d35-42a3-b0df-c619f0607c33 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 04:16:17.696308 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 04:16:17.696308 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 04:16:17.696308 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 04:16:17.696308 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 04:16:17.696308 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 04:16:17.696308 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 04:16:17.696308 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 04:16:17.696308 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 04:16:17.696308 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 04:16:17.696308 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 04:16:17.696308 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 04:16:17.696308 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 04:16:17.696308 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 04:16:17.696308 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 04:16:17.696308 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 04:16:17.697250 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 04:16:17.697250 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 04:16:17.697250 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 04:16:17.697250 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 04:16:17.697250 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 04:16:17.697250 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 04:16:17.697250 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 04:16:17.697250 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 04:16:17.697250 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 04:16:17.697250 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 04:16:17.697250 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 04:16:17.697250 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 04:16:17.697250 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 04:16:17.697250 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 04:16:17.697250 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 04:16:17.697250 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 04:16:17.698306 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 04:16:17.698306 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 04:16:17.698306 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 04:16:17.698306 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 04:16:17.698306 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 04:16:17.698306 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 04:16:17.698306 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 04:16:17.698306 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 04:16:17.698306 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 04:16:17.698306 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 04:16:17.698306 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 04:16:17.698306 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 04:16:17.698306 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 8/14] 10.222.0.30 () {56 vars in 1096 bytes} [Tue Sep 6 04:16:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 04:16:17.703218 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0bdc0cc-2d35-42a3-b0df-c619f0607c33 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:16:17.731021 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7a5afb-b897-4972-befb-c2b6f8b2ec86 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:16:17.733336 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:16:17.769956 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb058ef-7a23-4624-b877-d2e9231a1c71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:16:17.770847 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb058ef-7a23-4624-b877-d2e9231a1c71 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:16:17.771331 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb058ef-7a23-4624-b877-d2e9231a1c71 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:16:17.771895 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb058ef-7a23-4624-b877-d2e9231a1c71 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:16:17.786678 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:16:17.806264 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0bdc0cc-2d35-42a3-b0df-c619f0607c33 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:16:17.840638 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de05d78a-80e7-405a-a504-e22cd09d083f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:16:17.840866 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb058ef-7a23-4624-b877-d2e9231a1c71 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:16:17.841252 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de05d78a-80e7-405a-a504-e22cd09d083f None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:16:17.841564 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de05d78a-80e7-405a-a504-e22cd09d083f None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:16:17.841841 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de05d78a-80e7-405a-a504-e22cd09d083f None admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:16:17.842425 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb058ef-7a23-4624-b877-d2e9231a1c71 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:16:17.842509 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de05d78a-80e7-405a-a504-e22cd09d083f None admin] RBAC: Adding query filter params (name=admin) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:16:17.842804 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de05d78a-80e7-405a-a504-e22cd09d083f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:16:17.844185 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de05d78a-80e7-405a-a504-e22cd09d083f None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:16:17.856199 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 9/15] 10.222.0.30 () {56 vars in 1099 bytes} [Tue Sep 6 04:16:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 04:16:17.861694 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de05d78a-80e7-405a-a504-e22cd09d083f None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:16:17.939150 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 7/16] 10.222.0.30 () {58 vars in 1282 bytes} [Tue Sep 6 04:16:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 06 04:16:17.945166 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb058ef-7a23-4624-b877-d2e9231a1c71 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:16:17.947837 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:16:17.961144 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de05d78a-80e7-405a-a504-e22cd09d083f None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:16:17.996310 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6edd2604-5cb9-4d45-a67d-63432ae23c54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:16:17.996964 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edd2604-5cb9-4d45-a67d-63432ae23c54 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:16:17.997248 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edd2604-5cb9-4d45-a67d-63432ae23c54 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:16:17.997726 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edd2604-5cb9-4d45-a67d-63432ae23c54 None admin] PATH_INFO: `/v3/users/b7c5062264774b85bf945fb9ac7602db` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:16:17.998972 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6edd2604-5cb9-4d45-a67d-63432ae23c54 None admin] RBAC: Authorizing `identity:get_user(user_id=b7c5062264774b85bf945fb9ac7602db)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:16:18.000446 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6edd2604-5cb9-4d45-a67d-63432ae23c54 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:16:18.004268 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 10/17] 10.222.0.30 () {56 vars in 1177 bytes} [Tue Sep 6 04:16:17 2022] GET /identity/v3/users/b7c5062264774b85bf945fb9ac7602db => generated 247 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 04:16:18.010596 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6edd2604-5cb9-4d45-a67d-63432ae23c54 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:16:18.027088 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:16:18.054655 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb058ef-7a23-4624-b877-d2e9231a1c71 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:16:18.088566 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75f9ef5-1519-4be8-9221-b3cf6ab1ad8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:16:18.089450 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75f9ef5-1519-4be8-9221-b3cf6ab1ad8a None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:16:18.089732 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75f9ef5-1519-4be8-9221-b3cf6ab1ad8a None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:16:18.089999 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75f9ef5-1519-4be8-9221-b3cf6ab1ad8a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:16:18.108295 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75f9ef5-1519-4be8-9221-b3cf6ab1ad8a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:16:18.109907 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75f9ef5-1519-4be8-9221-b3cf6ab1ad8a None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:16:18.117819 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6edd2604-5cb9-4d45-a67d-63432ae23c54 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:16:18.128432 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-f75f9ef5-1519-4be8-9221-b3cf6ab1ad8a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 04:16:18.128432 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 04:16:18.128432 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 04:16:18.128432 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 04:16:18.128432 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 04:16:18.128432 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 04:16:18.128432 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 04:16:18.128432 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 04:16:18.128432 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 04:16:18.128432 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 04:16:18.128432 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 04:16:18.128432 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 04:16:18.128432 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 04:16:18.128432 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 04:16:18.128432 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 04:16:18.128432 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 04:16:18.129294 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 04:16:18.129294 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 04:16:18.129294 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 04:16:18.129294 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 04:16:18.129294 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 04:16:18.129294 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 04:16:18.129294 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 04:16:18.129294 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 04:16:18.129294 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 04:16:18.129294 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 04:16:18.129294 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 04:16:18.129294 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 04:16:18.129294 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 04:16:18.129294 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 04:16:18.129294 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 04:16:18.129294 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 04:16:18.129294 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 04:16:18.130133 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 04:16:18.130133 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 04:16:18.130133 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 04:16:18.130133 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 8/18] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Sep 6 04:16:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 04:16:18.135872 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75f9ef5-1519-4be8-9221-b3cf6ab1ad8a None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:16:18.151632 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b93082-24ce-4fd4-9d06-9230e6ab30b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:16:18.152263 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b93082-24ce-4fd4-9d06-9230e6ab30b4 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:16:18.152539 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b93082-24ce-4fd4-9d06-9230e6ab30b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:16:18.152810 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b93082-24ce-4fd4-9d06-9230e6ab30b4 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:16:18.153950 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b93082-24ce-4fd4-9d06-9230e6ab30b4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:16:18.155403 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b93082-24ce-4fd4-9d06-9230e6ab30b4 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:16:18.156927 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 11/19] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Sep 6 04:16:18 2022] GET /identity/v3/domains/default => generated 199 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 04:16:18.162522 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b93082-24ce-4fd4-9d06-9230e6ab30b4 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:16:18.191107 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:16:18.236741 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75f9ef5-1519-4be8-9221-b3cf6ab1ad8a None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:16:18.242069 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:16:18.261773 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b93082-24ce-4fd4-9d06-9230e6ab30b4 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:16:18.274134 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ae5fa1-b905-40bf-a673-adc94fae5a3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:16:18.274741 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ae5fa1-b905-40bf-a673-adc94fae5a3b None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:16:18.275018 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ae5fa1-b905-40bf-a673-adc94fae5a3b None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:16:18.275292 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ae5fa1-b905-40bf-a673-adc94fae5a3b None admin] PATH_INFO: `/v3/domains` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:16:18.275926 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ae5fa1-b905-40bf-a673-adc94fae5a3b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:16:18.276225 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ae5fa1-b905-40bf-a673-adc94fae5a3b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:16:18.277618 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ae5fa1-b905-40bf-a673-adc94fae5a3b None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:16:18.294954 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 9/20] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Sep 6 04:16:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 04:16:18.298499 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d50555-d174-444b-ad30-c6f2cd3acabb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:16:18.299157 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d50555-d174-444b-ad30-c6f2cd3acabb None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:16:18.299439 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d50555-d174-444b-ad30-c6f2cd3acabb None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:16:18.299717 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d50555-d174-444b-ad30-c6f2cd3acabb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:16:18.300471 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d50555-d174-444b-ad30-c6f2cd3acabb None admin] RBAC: Adding query filter params (role.id=ea4c814713b149108e488de855b025e4, scope.domain.id=default, user.id=b7c5062264774b85bf945fb9ac7602db) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:16:18.300782 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d50555-d174-444b-ad30-c6f2cd3acabb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:16:18.302306 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d50555-d174-444b-ad30-c6f2cd3acabb None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:16:18.311194 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 12/21] 10.222.0.30 () {56 vars in 1322 bytes} [Tue Sep 6 04:16:18 2022] GET /identity/v3/role_assignments?user.id=b7c5062264774b85bf945fb9ac7602db&scope.domain.id=default&role.id=ea4c814713b149108e488de855b025e4 => generated 233 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 04:16:20.433161 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae7f075-4ebc-4c9b-a753-93aa0b09abfe None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:16:20.434657 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae7f075-4ebc-4c9b-a753-93aa0b09abfe None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:16:20.435444 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae7f075-4ebc-4c9b-a753-93aa0b09abfe None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:16:20.438476 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 10/22] 10.222.0.30 () {54 vars in 927 bytes} [Tue Sep 6 04:16:20 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 04:16:20.446495 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd91ab71-2506-4498-acf7-40564bf01a8d None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:16:20.447438 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd91ab71-2506-4498-acf7-40564bf01a8d None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:16:20.447933 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd91ab71-2506-4498-acf7-40564bf01a8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:16:20.474760 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-fd91ab71-2506-4498-acf7-40564bf01a8d None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:16:20.477178 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-fd91ab71-2506-4498-acf7-40564bf01a8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:16:20.571167 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 13/23] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Sep 6 04:16:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 04:16:20.886790 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117e779c-1ba0-4849-8406-e5e0e6af422c None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:16:20.887198 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117e779c-1ba0-4849-8406-e5e0e6af422c None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:16:20.887381 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117e779c-1ba0-4849-8406-e5e0e6af422c None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:16:20.888768 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 11/24] 10.222.0.30 () {54 vars in 927 bytes} [Tue Sep 6 04:16:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 04:16:20.907239 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7803bfab-55ad-4f53-8c79-8761679d94dd None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:16:20.907552 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7803bfab-55ad-4f53-8c79-8761679d94dd None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:16:20.907816 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7803bfab-55ad-4f53-8c79-8761679d94dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:16:20.934329 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-7803bfab-55ad-4f53-8c79-8761679d94dd None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:16:20.935560 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-7803bfab-55ad-4f53-8c79-8761679d94dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:16:21.025166 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 14/25] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Sep 6 04:16:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 04:16:21.113676 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888e1d22-cb17-40b2-a672-4fc8c9bf473d None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:16:21.114323 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888e1d22-cb17-40b2-a672-4fc8c9bf473d None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:16:21.114813 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888e1d22-cb17-40b2-a672-4fc8c9bf473d None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:16:21.117018 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 12/26] 10.222.0.30 () {54 vars in 927 bytes} [Tue Sep 6 04:16:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 04:16:21.126059 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d150ced-1b72-44e2-8cce-eb3729562918 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:16:21.126630 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d150ced-1b72-44e2-8cce-eb3729562918 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:16:21.127122 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d150ced-1b72-44e2-8cce-eb3729562918 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:16:21.140427 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:16:21.155379 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-1d150ced-1b72-44e2-8cce-eb3729562918 None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:16:21.155765 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888e1d22-cb17-40b2-a672-4fc8c9bf473d None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:16:21.156651 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-1d150ced-1b72-44e2-8cce-eb3729562918 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:16:21.276647 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 15/27] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Sep 6 04:16:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 04:16:21.277932 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c16c815-6db2-4280-a709-1b38f40b0f04 None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:16:21.278221 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c16c815-6db2-4280-a709-1b38f40b0f04 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:16:21.278503 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c16c815-6db2-4280-a709-1b38f40b0f04 None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:16:21.279750 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 16/28] 10.222.0.30 () {54 vars in 927 bytes} [Tue Sep 6 04:16:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 04:16:21.284889 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6f419b-a936-434e-8bee-3f8a2fa5a5d3 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:16:21.285172 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6f419b-a936-434e-8bee-3f8a2fa5a5d3 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:16:21.285480 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6f419b-a936-434e-8bee-3f8a2fa5a5d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:16:21.325059 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-fa6f419b-a936-434e-8bee-3f8a2fa5a5d3 None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:16:21.325871 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888e1d22-cb17-40b2-a672-4fc8c9bf473d None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:16:21.328367 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-fa6f419b-a936-434e-8bee-3f8a2fa5a5d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:16:21.355295 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:16:21.396139 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1323498-f9bf-45f7-9880-edb44229d3de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:16:21.396139 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1323498-f9bf-45f7-9880-edb44229d3de None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:16:21.396139 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1323498-f9bf-45f7-9880-edb44229d3de None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:16:21.396139 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1323498-f9bf-45f7-9880-edb44229d3de None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:16:21.421690 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1323498-f9bf-45f7-9880-edb44229d3de None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 04:16:21.429902 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-f1323498-f9bf-45f7-9880-edb44229d3de None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 04:16:21.429902 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 04:16:21.429902 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 04:16:21.429902 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 04:16:21.429902 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 04:16:21.429902 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 04:16:21.429902 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 04:16:21.429902 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 04:16:21.429902 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 04:16:21.429902 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 04:16:21.429902 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 04:16:21.429902 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 04:16:21.429902 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 04:16:21.429902 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 04:16:21.429902 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 04:16:21.429902 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 04:16:21.430762 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 04:16:21.430762 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 04:16:21.430762 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 04:16:21.430762 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 04:16:21.430762 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 04:16:21.430762 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 04:16:21.430762 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 04:16:21.430762 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 04:16:21.430762 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 04:16:21.430762 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 04:16:21.430762 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 04:16:21.430762 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 04:16:21.430762 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 04:16:21.430762 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 04:16:21.430762 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 04:16:21.430762 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 04:16:21.432023 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 04:16:21.432023 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 04:16:21.432023 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 04:16:21.432023 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 04:16:21.432023 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 04:16:21.432023 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 04:16:21.432023 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 04:16:21.432023 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 04:16:21.432023 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 04:16:21.432023 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 04:16:21.432023 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 04:16:21.432023 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 04:16:21.432023 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 13/29] 10.222.0.30 () {56 vars in 1096 bytes} [Tue Sep 6 04:16:21 2022] GET /identity/v3/roles/admin => generated 83 bytes in 276 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 04:16:21.432023 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1323498-f9bf-45f7-9880-edb44229d3de None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:16:21.463037 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 17/30] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Sep 6 04:16:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 06 04:16:21.464611 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6f419b-a936-434e-8bee-3f8a2fa5a5d3 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:16:21.565675 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:16:21.618486 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6f419b-a936-434e-8bee-3f8a2fa5a5d3 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:16:21.632863 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1323498-f9bf-45f7-9880-edb44229d3de None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:16:21.682829 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca56ce81-712a-4540-a164-59c3254bb9c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:16:21.683386 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca56ce81-712a-4540-a164-59c3254bb9c1 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:16:21.683671 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca56ce81-712a-4540-a164-59c3254bb9c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:16:21.683952 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca56ce81-712a-4540-a164-59c3254bb9c1 None admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:16:21.684640 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca56ce81-712a-4540-a164-59c3254bb9c1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:16:21.684950 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca56ce81-712a-4540-a164-59c3254bb9c1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:16:21.687261 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca56ce81-712a-4540-a164-59c3254bb9c1 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:16:21.696828 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b92e24-1305-42ac-8bc3-5cc09548dc10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:16:21.697487 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b92e24-1305-42ac-8bc3-5cc09548dc10 None admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:16:21.697772 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b92e24-1305-42ac-8bc3-5cc09548dc10 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:16:21.698049 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b92e24-1305-42ac-8bc3-5cc09548dc10 None admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:16:21.698740 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b92e24-1305-42ac-8bc3-5cc09548dc10 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:16:21.700159 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b92e24-1305-42ac-8bc3-5cc09548dc10 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:16:21.709106 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 18/31] 10.222.0.30 () {56 vars in 1099 bytes} [Tue Sep 6 04:16:21 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:16:21.712321 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca56ce81-712a-4540-a164-59c3254bb9c1 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:16:21.822331 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca56ce81-712a-4540-a164-59c3254bb9c1 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:16:21.851611 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:16:21.869789 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83fdb772-797c-4bab-8d91-79471e5798bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:16:21.870767 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fdb772-797c-4bab-8d91-79471e5798bd None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:16:21.871330 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fdb772-797c-4bab-8d91-79471e5798bd None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:16:21.871800 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fdb772-797c-4bab-8d91-79471e5798bd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:16:21.884058 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83fdb772-797c-4bab-8d91-79471e5798bd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:16:21.885876 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83fdb772-797c-4bab-8d91-79471e5798bd None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:16:21.904416 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-83fdb772-797c-4bab-8d91-79471e5798bd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 04:16:21.904416 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 04:16:21.904416 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 04:16:21.904416 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 04:16:21.904416 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 04:16:21.904416 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 04:16:21.904416 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 04:16:21.904416 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 04:16:21.904416 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 04:16:21.904416 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 04:16:21.904416 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 04:16:21.904416 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 04:16:21.904416 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 04:16:21.904416 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 04:16:21.904416 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 04:16:21.904416 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 04:16:21.905289 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 04:16:21.905289 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 04:16:21.905289 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 04:16:21.905289 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 04:16:21.905289 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 04:16:21.905289 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 04:16:21.905289 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 04:16:21.905289 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 04:16:21.905289 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 04:16:21.905289 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 04:16:21.905289 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 04:16:21.905289 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 04:16:21.905289 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 04:16:21.905289 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 04:16:21.905289 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 04:16:21.905289 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 04:16:21.905289 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 04:16:21.906551 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 04:16:21.906551 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 04:16:21.906551 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 04:16:21.906551 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 19/32] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Sep 6 04:16:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 04:16:21.906551 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83fdb772-797c-4bab-8d91-79471e5798bd None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:16:21.973565 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 14/33] 10.222.0.30 () {60 vars in 1131 bytes} [Tue Sep 6 04:16:21 2022] POST /identity/v3/roles => generated 225 bytes in 536 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 04:16:21.973565 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b92e24-1305-42ac-8bc3-5cc09548dc10 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:16:22.057768 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:16:22.072277 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83fdb772-797c-4bab-8d91-79471e5798bd None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:16:22.118886 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b92e24-1305-42ac-8bc3-5cc09548dc10 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:16:22.140469 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db4c904-afb8-403d-aae2-7c5634897729 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:16:22.141127 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db4c904-afb8-403d-aae2-7c5634897729 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:16:22.141498 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db4c904-afb8-403d-aae2-7c5634897729 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:16:22.141803 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db4c904-afb8-403d-aae2-7c5634897729 None admin] PATH_INFO: `/v3/users/b7c5062264774b85bf945fb9ac7602db` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:16:22.143023 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db4c904-afb8-403d-aae2-7c5634897729 None admin] RBAC: Authorizing `identity:get_user(user_id=b7c5062264774b85bf945fb9ac7602db)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:16:22.144526 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db4c904-afb8-403d-aae2-7c5634897729 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:16:22.146308 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 20/34] 10.222.0.30 () {56 vars in 1177 bytes} [Tue Sep 6 04:16:21 2022] GET /identity/v3/users/b7c5062264774b85bf945fb9ac7602db => generated 247 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:16:22.147179 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db4c904-afb8-403d-aae2-7c5634897729 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:16:22.162009 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f3d489-852a-4fbb-9ebd-e107621280fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:16:22.162733 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f3d489-852a-4fbb-9ebd-e107621280fe None admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:16:22.164040 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f3d489-852a-4fbb-9ebd-e107621280fe None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:16:22.164396 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f3d489-852a-4fbb-9ebd-e107621280fe None admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:16:22.165258 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f3d489-852a-4fbb-9ebd-e107621280fe None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:16:22.166937 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f3d489-852a-4fbb-9ebd-e107621280fe None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:16:22.197989 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:16:22.251917 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db4c904-afb8-403d-aae2-7c5634897729 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:16:22.272040 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 15/35] 10.222.0.30 () {60 vars in 1131 bytes} [Tue Sep 6 04:16:21 2022] POST /identity/v3/roles => generated 219 bytes in 302 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 04:16:22.273486 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f126437d-c419-44c5-917f-cdc938970c52 None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:16:22.279654 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f126437d-c419-44c5-917f-cdc938970c52 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:16:22.279654 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f126437d-c419-44c5-917f-cdc938970c52 None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:16:22.279654 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 16/36] 10.222.0.30 () {54 vars in 927 bytes} [Tue Sep 6 04:16:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 04:16:22.279654 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8b0ad5-6427-4c47-b13f-dd9e68b54128 None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:16:22.279654 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8b0ad5-6427-4c47-b13f-dd9e68b54128 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:16:22.279654 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8b0ad5-6427-4c47-b13f-dd9e68b54128 None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:16:22.279654 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 17/37] 10.222.0.30 () {54 vars in 927 bytes} [Tue Sep 6 04:16:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 04:16:22.280777 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8b0ad5-6427-4c47-b13f-dd9e68b54128 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:16:22.296700 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3acb416-a8f2-4a0e-8868-ac57350dedb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:16:22.297360 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3acb416-a8f2-4a0e-8868-ac57350dedb9 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:16:22.297650 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3acb416-a8f2-4a0e-8868-ac57350dedb9 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:16:22.297930 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3acb416-a8f2-4a0e-8868-ac57350dedb9 None admin] PATH_INFO: `/v3/domains` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:16:22.298608 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3acb416-a8f2-4a0e-8868-ac57350dedb9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:16:22.298917 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3acb416-a8f2-4a0e-8868-ac57350dedb9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:16:22.300371 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3acb416-a8f2-4a0e-8868-ac57350dedb9 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:16:22.321374 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 21/38] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Sep 6 04:16:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:16:22.322195 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baad56a6-f275-4421-885f-fec46703969c None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:16:22.322490 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baad56a6-f275-4421-885f-fec46703969c None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:16:22.322768 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baad56a6-f275-4421-885f-fec46703969c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:16:22.342713 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-baad56a6-f275-4421-885f-fec46703969c None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:16:22.343964 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-baad56a6-f275-4421-885f-fec46703969c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:16:22.405539 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:16:22.412766 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8b0ad5-6427-4c47-b13f-dd9e68b54128 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:16:22.436003 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 22/39] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Sep 6 04:16:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 04:16:22.436924 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd9f45f-b41e-490d-a6db-43f63f89ad87 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:16:22.437281 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd9f45f-b41e-490d-a6db-43f63f89ad87 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:16:22.437524 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd9f45f-b41e-490d-a6db-43f63f89ad87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:16:22.459004 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c179ea-02a0-4992-a23b-94541a45e19c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:16:22.459788 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c179ea-02a0-4992-a23b-94541a45e19c None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:16:22.460404 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c179ea-02a0-4992-a23b-94541a45e19c None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:16:22.460567 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-1cd9f45f-b41e-490d-a6db-43f63f89ad87 None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:16:22.460672 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c179ea-02a0-4992-a23b-94541a45e19c None admin] PATH_INFO: `/v3/domains/default` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:16:22.461835 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-1cd9f45f-b41e-490d-a6db-43f63f89ad87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:16:22.463085 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c179ea-02a0-4992-a23b-94541a45e19c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:16:22.464251 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c179ea-02a0-4992-a23b-94541a45e19c None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:16:22.476786 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 18/40] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Sep 6 04:16:22 2022] GET /identity/v3/domains/default => generated 199 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:16:22.476786 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c179ea-02a0-4992-a23b-94541a45e19c None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:16:22.503596 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:16:22.557339 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 23/41] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Sep 6 04:16:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 04:16:22.557339 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd9f45f-b41e-490d-a6db-43f63f89ad87 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:16:22.595582 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c179ea-02a0-4992-a23b-94541a45e19c None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:16:22.631012 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41a7cc6-5c63-487b-9c39-bb4269027a5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:16:22.631689 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41a7cc6-5c63-487b-9c39-bb4269027a5c None admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:16:22.632051 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41a7cc6-5c63-487b-9c39-bb4269027a5c None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:16:22.632425 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41a7cc6-5c63-487b-9c39-bb4269027a5c None admin] PATH_INFO: `/v3/projects` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:16:22.633218 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41a7cc6-5c63-487b-9c39-bb4269027a5c None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:16:22.634703 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41a7cc6-5c63-487b-9c39-bb4269027a5c None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:16:22.656371 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd9f45f-b41e-490d-a6db-43f63f89ad87 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:16:22.692084 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799d72c1-4ec7-4d6e-80a6-78b1c277ef73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:16:22.692741 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799d72c1-4ec7-4d6e-80a6-78b1c277ef73 None admin] REQUEST_METHOD: `PUT` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:16:22.693019 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799d72c1-4ec7-4d6e-80a6-78b1c277ef73 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:16:22.693289 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799d72c1-4ec7-4d6e-80a6-78b1c277ef73 None admin] PATH_INFO: `/v3/domains/default/users/b7c5062264774b85bf945fb9ac7602db/roles/ea4c814713b149108e488de855b025e4` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:16:22.695384 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799d72c1-4ec7-4d6e-80a6-78b1c277ef73 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=b7c5062264774b85bf945fb9ac7602db, role_id=ea4c814713b149108e488de855b025e4)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:16:22.696904 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799d72c1-4ec7-4d6e-80a6-78b1c277ef73 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:16:22.754481 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 19/42] 10.222.0.30 () {60 vars in 1141 bytes} [Tue Sep 6 04:16:22 2022] POST /identity/v3/projects => generated 301 bytes in 281 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 04:16:22.755218 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41a7cc6-5c63-487b-9c39-bb4269027a5c None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:16:22.779792 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:16:22.881396 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41a7cc6-5c63-487b-9c39-bb4269027a5c None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:16:22.926626 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebd107c-c943-4c8f-beba-ea0c059d0069 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:16:22.927271 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebd107c-c943-4c8f-beba-ea0c059d0069 None admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:16:22.927551 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebd107c-c943-4c8f-beba-ea0c059d0069 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:16:22.927829 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebd107c-c943-4c8f-beba-ea0c059d0069 None admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:16:22.928524 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ebd107c-c943-4c8f-beba-ea0c059d0069 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:16:22.930166 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ebd107c-c943-4c8f-beba-ea0c059d0069 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:16:23.003591 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 24/43] 10.222.0.30 () {58 vars in 1361 bytes} [Tue Sep 6 04:16:22 2022] PUT /identity/v3/domains/default/users/b7c5062264774b85bf945fb9ac7602db/roles/ea4c814713b149108e488de855b025e4 => generated 0 bytes in 451 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 04:16:23.004954 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799d72c1-4ec7-4d6e-80a6-78b1c277ef73 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:16:23.035761 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 20/44] 10.222.0.30 () {60 vars in 1131 bytes} [Tue Sep 6 04:16:22 2022] POST /identity/v3/roles => generated 223 bytes in 281 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 04:16:23.039081 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:16:23.129774 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799d72c1-4ec7-4d6e-80a6-78b1c277ef73 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:16:23.173856 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677fad06-7ffb-4d91-930d-7be534f2e66c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:16:23.174523 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677fad06-7ffb-4d91-930d-7be534f2e66c None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:16:23.174805 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677fad06-7ffb-4d91-930d-7be534f2e66c None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:16:23.175078 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677fad06-7ffb-4d91-930d-7be534f2e66c None admin] PATH_INFO: `/v3/domains/default` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:16:23.176342 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-677fad06-7ffb-4d91-930d-7be534f2e66c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:16:23.178067 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-677fad06-7ffb-4d91-930d-7be534f2e66c None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:16:23.183156 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 25/45] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Sep 6 04:16:23 2022] GET /identity/v3/domains/default => generated 199 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:16:23.189861 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebd107c-c943-4c8f-beba-ea0c059d0069 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:16:23.275878 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:16:23.297786 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebd107c-c943-4c8f-beba-ea0c059d0069 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:16:23.330255 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3a46db-222d-479f-9611-3c04798ea320 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:16:23.330871 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3a46db-222d-479f-9611-3c04798ea320 None admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:16:23.331152 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3a46db-222d-479f-9611-3c04798ea320 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:16:23.331423 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3a46db-222d-479f-9611-3c04798ea320 None admin] PATH_INFO: `/v3/projects` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:16:23.332083 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3a46db-222d-479f-9611-3c04798ea320 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:16:23.333584 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3a46db-222d-479f-9611-3c04798ea320 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:16:23.439916 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 21/46] 10.222.0.30 () {60 vars in 1141 bytes} [Tue Sep 6 04:16:23 2022] POST /identity/v3/projects => generated 312 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 04:16:24.930879 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7f665a-35ce-46a8-a835-98ba38926629 None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:16:24.931428 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7f665a-35ce-46a8-a835-98ba38926629 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:16:24.931428 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7f665a-35ce-46a8-a835-98ba38926629 None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:16:24.932781 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 26/47] 10.222.0.30 () {54 vars in 927 bytes} [Tue Sep 6 04:16:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 04:16:24.938164 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44a58ba-2e42-4fa5-8609-8b76fc8ca487 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:16:24.938588 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44a58ba-2e42-4fa5-8609-8b76fc8ca487 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:16:24.938861 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44a58ba-2e42-4fa5-8609-8b76fc8ca487 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:16:24.957670 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-d44a58ba-2e42-4fa5-8609-8b76fc8ca487 None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:16:24.959196 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-d44a58ba-2e42-4fa5-8609-8b76fc8ca487 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:16:25.031969 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 22/48] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Sep 6 04:16:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 04:16:25.302646 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7f665a-35ce-46a8-a835-98ba38926629 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:16:25.366254 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddad390-b2b7-4220-9a77-dab95f995616 None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:16:25.366540 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddad390-b2b7-4220-9a77-dab95f995616 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:16:25.366820 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddad390-b2b7-4220-9a77-dab95f995616 None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:16:25.368307 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 23/49] 10.222.0.30 () {54 vars in 927 bytes} [Tue Sep 6 04:16:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 04:16:25.372841 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d3dceb-b43f-496e-ba62-9e2074a595aa None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:16:25.373127 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d3dceb-b43f-496e-ba62-9e2074a595aa None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:16:25.373428 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d3dceb-b43f-496e-ba62-9e2074a595aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:16:25.394948 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-24d3dceb-b43f-496e-ba62-9e2074a595aa None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:16:25.396132 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-24d3dceb-b43f-496e-ba62-9e2074a595aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:16:25.402370 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:16:25.409824 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7f665a-35ce-46a8-a835-98ba38926629 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:16:25.443854 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328e82f6-37c2-468a-99ba-e3b5624560d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:16:25.444605 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328e82f6-37c2-468a-99ba-e3b5624560d7 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:16:25.444967 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328e82f6-37c2-468a-99ba-e3b5624560d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:16:25.445322 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328e82f6-37c2-468a-99ba-e3b5624560d7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:16:25.448981 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:16:25.456004 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-328e82f6-37c2-468a-99ba-e3b5624560d7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 04:16:25.461805 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-328e82f6-37c2-468a-99ba-e3b5624560d7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 04:16:25.461805 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 04:16:25.461805 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 04:16:25.461805 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 04:16:25.461805 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 04:16:25.461805 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 04:16:25.461805 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 04:16:25.461805 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 04:16:25.461805 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 04:16:25.461805 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 04:16:25.461805 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 04:16:25.461805 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 04:16:25.461805 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 04:16:25.461805 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 04:16:25.461805 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 04:16:25.461805 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 04:16:25.462853 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 04:16:25.462853 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 04:16:25.462853 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 04:16:25.462853 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 04:16:25.462853 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 04:16:25.462853 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 04:16:25.462853 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 04:16:25.462853 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 04:16:25.462853 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 04:16:25.462853 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 04:16:25.462853 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 04:16:25.462853 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 04:16:25.462853 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 04:16:25.462853 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 04:16:25.462853 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 04:16:25.462853 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 04:16:25.464051 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 04:16:25.464051 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 04:16:25.464051 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 04:16:25.464051 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 04:16:25.464051 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 04:16:25.464051 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 04:16:25.464051 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 04:16:25.464051 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 04:16:25.464051 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 04:16:25.464051 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 04:16:25.464051 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 04:16:25.464051 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 04:16:25.464699 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 24/50] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Sep 6 04:16:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 06 04:16:25.464766 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 27/51] 10.222.0.30 () {56 vars in 1096 bytes} [Tue Sep 6 04:16:25 2022] GET /identity/v3/roles/admin => generated 83 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 04:16:25.470515 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d3dceb-b43f-496e-ba62-9e2074a595aa None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:16:25.569621 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d3dceb-b43f-496e-ba62-9e2074a595aa None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:16:25.604445 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ef798b-7069-434d-819e-2f29865730d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:16:25.605061 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ef798b-7069-434d-819e-2f29865730d9 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:16:25.605373 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ef798b-7069-434d-819e-2f29865730d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:16:25.605660 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ef798b-7069-434d-819e-2f29865730d9 None admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:16:25.606331 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ef798b-7069-434d-819e-2f29865730d9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:16:25.606636 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ef798b-7069-434d-819e-2f29865730d9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:16:25.608016 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ef798b-7069-434d-819e-2f29865730d9 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:16:25.619659 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 25/52] 10.222.0.30 () {56 vars in 1099 bytes} [Tue Sep 6 04:16:25 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 04:16:25.625250 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328e82f6-37c2-468a-99ba-e3b5624560d7 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:16:25.722338 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328e82f6-37c2-468a-99ba-e3b5624560d7 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:16:25.741572 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ef798b-7069-434d-819e-2f29865730d9 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:16:25.756386 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf0a009-d467-4a00-a591-8be48d723013 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:16:25.757601 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf0a009-d467-4a00-a591-8be48d723013 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:16:25.757883 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf0a009-d467-4a00-a591-8be48d723013 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:16:25.758163 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf0a009-d467-4a00-a591-8be48d723013 None admin] PATH_INFO: `/v3/users/b7c5062264774b85bf945fb9ac7602db` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:16:25.759399 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf0a009-d467-4a00-a591-8be48d723013 None admin] RBAC: Authorizing `identity:get_user(user_id=b7c5062264774b85bf945fb9ac7602db)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:16:25.760879 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf0a009-d467-4a00-a591-8be48d723013 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:16:25.762527 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 28/53] 10.222.0.30 () {56 vars in 1177 bytes} [Tue Sep 6 04:16:25 2022] GET /identity/v3/users/b7c5062264774b85bf945fb9ac7602db => generated 247 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 04:16:25.767966 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf0a009-d467-4a00-a591-8be48d723013 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:16:25.791825 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:16:25.793814 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:16:25.842690 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ef798b-7069-434d-819e-2f29865730d9 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:16:25.866928 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf0a009-d467-4a00-a591-8be48d723013 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:16:25.877260 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169587c2-af48-4b4d-851d-d75e986a653b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:16:25.878186 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169587c2-af48-4b4d-851d-d75e986a653b None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:16:25.878461 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169587c2-af48-4b4d-851d-d75e986a653b None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:16:25.878732 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169587c2-af48-4b4d-851d-d75e986a653b None admin] PATH_INFO: `/v3/domains/default` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:16:25.879832 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-169587c2-af48-4b4d-851d-d75e986a653b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:16:25.881276 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-169587c2-af48-4b4d-851d-d75e986a653b None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:16:25.883596 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 26/54] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Sep 6 04:16:25 2022] GET /identity/v3/domains/default => generated 199 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 04:16:25.888951 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169587c2-af48-4b4d-851d-d75e986a653b None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:16:25.901603 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a01e32-dcbd-48b5-a745-e32d7e616dc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:16:25.902247 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a01e32-dcbd-48b5-a745-e32d7e616dc0 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:16:25.902533 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a01e32-dcbd-48b5-a745-e32d7e616dc0 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:16:25.902816 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a01e32-dcbd-48b5-a745-e32d7e616dc0 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:16:25.904079 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a01e32-dcbd-48b5-a745-e32d7e616dc0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:16:25.905446 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a01e32-dcbd-48b5-a745-e32d7e616dc0 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:16:25.907027 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 29/55] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Sep 6 04:16:25 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 04:16:25.912568 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a01e32-dcbd-48b5-a745-e32d7e616dc0 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:16:25.987881 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169587c2-af48-4b4d-851d-d75e986a653b None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:16:26.002056 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:16:26.006161 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:16:26.012445 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a01e32-dcbd-48b5-a745-e32d7e616dc0 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:16:26.023241 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c57115-3dc8-4809-806e-6a2c6514f3d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:16:26.023846 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c57115-3dc8-4809-806e-6a2c6514f3d3 None admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:16:26.024124 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c57115-3dc8-4809-806e-6a2c6514f3d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:16:26.024395 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c57115-3dc8-4809-806e-6a2c6514f3d3 None admin] PATH_INFO: `/v3/projects` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:16:26.025042 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c57115-3dc8-4809-806e-6a2c6514f3d3 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:16:26.026523 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c57115-3dc8-4809-806e-6a2c6514f3d3 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:16:26.047560 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9b8f78-f156-4954-805d-36f83e70c71b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:16:26.048172 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9b8f78-f156-4954-805d-36f83e70c71b None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:16:26.048483 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9b8f78-f156-4954-805d-36f83e70c71b None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:16:26.048775 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9b8f78-f156-4954-805d-36f83e70c71b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:16:26.049593 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9b8f78-f156-4954-805d-36f83e70c71b None admin] RBAC: Adding query filter params (role.id=ea4c814713b149108e488de855b025e4, scope.domain.id=default, user.id=b7c5062264774b85bf945fb9ac7602db) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:16:26.049906 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9b8f78-f156-4954-805d-36f83e70c71b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:16:26.051352 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9b8f78-f156-4954-805d-36f83e70c71b None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:16:26.057652 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 30/56] 10.222.0.30 () {56 vars in 1322 bytes} [Tue Sep 6 04:16:25 2022] GET /identity/v3/role_assignments?user.id=b7c5062264774b85bf945fb9ac7602db&scope.domain.id=default&role.id=ea4c814713b149108e488de855b025e4 => generated 530 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 04:16:26.140061 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 27/57] 10.222.0.30 () {60 vars in 1140 bytes} [Tue Sep 6 04:16:25 2022] POST /identity/v3/projects => generated 298 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 04:16:28.052197 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b579b0-3506-48e3-a014-24137e2315f8 None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:16:28.052712 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b579b0-3506-48e3-a014-24137e2315f8 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:16:28.052800 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b579b0-3506-48e3-a014-24137e2315f8 None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:16:28.054205 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 31/58] 10.222.0.30 () {54 vars in 927 bytes} [Tue Sep 6 04:16:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 04:16:28.059831 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375d4c04-5b63-4df8-8fc0-d12d86a7990d None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:16:28.060263 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375d4c04-5b63-4df8-8fc0-d12d86a7990d None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:16:28.060388 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375d4c04-5b63-4df8-8fc0-d12d86a7990d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:16:28.180893 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-375d4c04-5b63-4df8-8fc0-d12d86a7990d None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:16:28.182139 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-375d4c04-5b63-4df8-8fc0-d12d86a7990d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:16:28.246195 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 28/59] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Sep 6 04:16:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 04:16:28.515707 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b579b0-3506-48e3-a014-24137e2315f8 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:16:28.615819 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b579b0-3506-48e3-a014-24137e2315f8 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:16:28.649106 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1549d1fb-4e69-438b-8987-93a54276b6d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:16:28.649701 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1549d1fb-4e69-438b-8987-93a54276b6d6 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:16:28.657369 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1549d1fb-4e69-438b-8987-93a54276b6d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:16:28.657369 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1549d1fb-4e69-438b-8987-93a54276b6d6 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:16:28.657369 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1549d1fb-4e69-438b-8987-93a54276b6d6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:16:28.657369 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1549d1fb-4e69-438b-8987-93a54276b6d6 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:16:28.657369 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 32/60] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Sep 6 04:16:28 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:16:28.659832 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375d4c04-5b63-4df8-8fc0-d12d86a7990d None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:16:28.753359 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:16:28.756195 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375d4c04-5b63-4df8-8fc0-d12d86a7990d None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:16:28.792951 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313fa63e-ce1a-4477-90a9-0c7ab7074b19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:16:28.793597 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313fa63e-ce1a-4477-90a9-0c7ab7074b19 None admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:16:28.793883 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313fa63e-ce1a-4477-90a9-0c7ab7074b19 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:16:28.794158 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313fa63e-ce1a-4477-90a9-0c7ab7074b19 None admin] PATH_INFO: `/v3/users` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:16:28.794816 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313fa63e-ce1a-4477-90a9-0c7ab7074b19 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:16:28.796286 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313fa63e-ce1a-4477-90a9-0c7ab7074b19 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:16:28.899456 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 29/61] 10.222.0.30 () {60 vars in 1132 bytes} [Tue Sep 6 04:16:28 2022] POST /identity/v3/users => generated 275 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 04:16:31.956522 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938713f7-92c2-4ef3-91a0-b3dd2524a9b6 None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:16:31.957022 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938713f7-92c2-4ef3-91a0-b3dd2524a9b6 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:16:31.957093 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938713f7-92c2-4ef3-91a0-b3dd2524a9b6 None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:16:31.958447 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 33/62] 10.222.0.30 () {54 vars in 927 bytes} [Tue Sep 6 04:16:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 04:16:31.963977 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8a7b0e-7a1c-409c-be6d-2e26a18f575b None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:16:31.964788 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8a7b0e-7a1c-409c-be6d-2e26a18f575b None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:16:31.965255 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8a7b0e-7a1c-409c-be6d-2e26a18f575b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:16:31.977409 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7b5e71-2831-4de8-a19e-827fddc4013e None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:16:31.978077 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7b5e71-2831-4de8-a19e-827fddc4013e None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:16:31.978544 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7b5e71-2831-4de8-a19e-827fddc4013e None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:16:31.980336 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 34/63] 10.222.0.30 () {54 vars in 927 bytes} [Tue Sep 6 04:16:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 04:16:31.986036 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e033c71b-a55d-4199-901e-907f0a2c6af5 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:16:31.986551 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e033c71b-a55d-4199-901e-907f0a2c6af5 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:16:31.987035 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e033c71b-a55d-4199-901e-907f0a2c6af5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:16:31.990355 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-5b8a7b0e-7a1c-409c-be6d-2e26a18f575b None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:16:31.992694 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-5b8a7b0e-7a1c-409c-be6d-2e26a18f575b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:16:32.012077 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-e033c71b-a55d-4199-901e-907f0a2c6af5 None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:16:32.014992 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-e033c71b-a55d-4199-901e-907f0a2c6af5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:16:32.061113 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:16:32.127961 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 30/64] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Sep 6 04:16:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 06 04:16:32.148276 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 35/65] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Sep 6 04:16:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 04:16:32.259367 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519d049b-3f93-447a-9e86-6ea757a0699d None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:16:32.259688 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519d049b-3f93-447a-9e86-6ea757a0699d None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:16:32.259844 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519d049b-3f93-447a-9e86-6ea757a0699d None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:16:32.261385 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 31/66] 10.222.0.30 () {54 vars in 927 bytes} [Tue Sep 6 04:16:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 04:16:32.275558 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b6022d-0ecc-4dd9-88a9-85308c2735c8 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:16:32.276015 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b6022d-0ecc-4dd9-88a9-85308c2735c8 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:16:32.276437 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b6022d-0ecc-4dd9-88a9-85308c2735c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:16:32.294919 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-50b6022d-0ecc-4dd9-88a9-85308c2735c8 None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:16:32.296415 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-50b6022d-0ecc-4dd9-88a9-85308c2735c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:16:32.367600 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 36/67] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Sep 6 04:16:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 04:16:32.471519 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519d049b-3f93-447a-9e86-6ea757a0699d None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:16:32.522659 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6318f855-2961-4d8f-bc8e-a1e732c0afcc None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:16:32.523239 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6318f855-2961-4d8f-bc8e-a1e732c0afcc None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:16:32.523725 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6318f855-2961-4d8f-bc8e-a1e732c0afcc None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:16:32.525357 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 37/68] 10.222.0.30 () {54 vars in 927 bytes} [Tue Sep 6 04:16:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 06 04:16:32.530884 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35afe342-001d-43d6-a87e-7e8e074a3e1b None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:16:32.531404 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35afe342-001d-43d6-a87e-7e8e074a3e1b None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:16:32.531867 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35afe342-001d-43d6-a87e-7e8e074a3e1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:16:32.551517 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-35afe342-001d-43d6-a87e-7e8e074a3e1b None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:16:32.553083 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-35afe342-001d-43d6-a87e-7e8e074a3e1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:16:32.601854 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:16:32.614727 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519d049b-3f93-447a-9e86-6ea757a0699d None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:16:32.627027 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 38/69] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Sep 6 04:16:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 06 04:16:32.629637 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35afe342-001d-43d6-a87e-7e8e074a3e1b None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:16:32.644192 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:16:32.672759 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee754f7-59ed-410d-88e6-db2381e9505b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:16:32.674456 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee754f7-59ed-410d-88e6-db2381e9505b None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:16:32.675142 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee754f7-59ed-410d-88e6-db2381e9505b None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:16:32.675868 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee754f7-59ed-410d-88e6-db2381e9505b None admin] PATH_INFO: `/v3/roles/member` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:16:32.695660 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee754f7-59ed-410d-88e6-db2381e9505b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 04:16:32.706330 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-7ee754f7-59ed-410d-88e6-db2381e9505b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 06 04:16:32.706330 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 04:16:32.706330 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 04:16:32.706330 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 04:16:32.706330 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 04:16:32.706330 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 04:16:32.706330 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 04:16:32.706330 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 04:16:32.706330 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 04:16:32.706330 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 04:16:32.706330 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 04:16:32.706330 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 04:16:32.706330 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 04:16:32.706330 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 04:16:32.706330 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 04:16:32.706330 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 04:16:32.707233 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 04:16:32.707233 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 04:16:32.707233 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 04:16:32.707233 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 04:16:32.707233 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 04:16:32.707233 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 04:16:32.707233 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 04:16:32.707233 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 04:16:32.707233 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 04:16:32.707233 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 04:16:32.707233 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 04:16:32.707233 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 04:16:32.707233 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 04:16:32.707233 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 04:16:32.707233 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 04:16:32.707233 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 04:16:32.707991 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 04:16:32.707991 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 04:16:32.707991 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 04:16:32.707991 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 04:16:32.707991 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 04:16:32.707991 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 04:16:32.707991 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 04:16:32.707991 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 04:16:32.707991 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 04:16:32.707991 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 04:16:32.707991 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 06 04:16:32.707991 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 04:16:32.712336 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 32/70] 10.222.0.30 () {56 vars in 1099 bytes} [Tue Sep 6 04:16:32 2022] GET /identity/v3/roles/member => generated 84 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 04:16:32.716883 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09baad51-a529-42ab-bca9-121d200080f0 None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:16:32.717382 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09baad51-a529-42ab-bca9-121d200080f0 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:16:32.717853 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09baad51-a529-42ab-bca9-121d200080f0 None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:16:32.719357 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 33/71] 10.222.0.30 () {54 vars in 927 bytes} [Tue Sep 6 04:16:32 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 04:16:32.720439 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09baad51-a529-42ab-bca9-121d200080f0 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:16:32.749211 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35afe342-001d-43d6-a87e-7e8e074a3e1b None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:16:32.784696 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6edaae-f6c3-4996-a101-b46f344e4afb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:16:32.785262 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6edaae-f6c3-4996-a101-b46f344e4afb None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:16:32.785576 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6edaae-f6c3-4996-a101-b46f344e4afb None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:16:32.785852 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6edaae-f6c3-4996-a101-b46f344e4afb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:16:32.796632 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6edaae-f6c3-4996-a101-b46f344e4afb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 04:16:32.801498 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-0d6edaae-f6c3-4996-a101-b46f344e4afb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 04:16:32.801498 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 04:16:32.801498 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 04:16:32.801498 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 04:16:32.801498 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 04:16:32.801498 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 04:16:32.801498 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 04:16:32.801498 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 04:16:32.801498 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 04:16:32.801498 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 04:16:32.801498 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 04:16:32.801498 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 04:16:32.801498 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 04:16:32.801498 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 04:16:32.801498 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 04:16:32.801498 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 04:16:32.802501 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 04:16:32.802501 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 04:16:32.802501 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 04:16:32.802501 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 04:16:32.802501 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 04:16:32.802501 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 04:16:32.802501 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 04:16:32.802501 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 04:16:32.802501 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 04:16:32.802501 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 04:16:32.802501 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 04:16:32.802501 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 04:16:32.802501 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 04:16:32.802501 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 04:16:32.802501 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 04:16:32.802501 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 04:16:32.803950 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 04:16:32.803950 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 04:16:32.803950 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 04:16:32.803950 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 04:16:32.803950 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 04:16:32.803950 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 04:16:32.803950 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 04:16:32.803950 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 04:16:32.803950 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 04:16:32.803950 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 04:16:32.803950 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 04:16:32.803950 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 04:16:32.803950 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 39/72] 10.222.0.30 () {56 vars in 1096 bytes} [Tue Sep 6 04:16:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 04:16:32.803950 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6edaae-f6c3-4996-a101-b46f344e4afb None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:16:32.837197 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09baad51-a529-42ab-bca9-121d200080f0 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:16:32.873950 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833cf452-a715-426e-bf16-7ebc0dc5b5cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:16:32.874587 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833cf452-a715-426e-bf16-7ebc0dc5b5cd None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:16:32.874867 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833cf452-a715-426e-bf16-7ebc0dc5b5cd None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:16:32.875139 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833cf452-a715-426e-bf16-7ebc0dc5b5cd None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:16:32.878235 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:16:32.886094 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-833cf452-a715-426e-bf16-7ebc0dc5b5cd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 04:16:32.890929 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-833cf452-a715-426e-bf16-7ebc0dc5b5cd None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 06 04:16:32.890929 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 04:16:32.890929 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 04:16:32.890929 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 04:16:32.890929 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 04:16:32.890929 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 04:16:32.890929 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 04:16:32.890929 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 04:16:32.890929 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 04:16:32.890929 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 04:16:32.890929 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 04:16:32.890929 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 04:16:32.890929 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 04:16:32.890929 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 04:16:32.890929 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 04:16:32.890929 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 04:16:32.891827 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 04:16:32.891827 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 04:16:32.891827 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 04:16:32.891827 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 04:16:32.891827 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 04:16:32.891827 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 04:16:32.891827 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 04:16:32.891827 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 04:16:32.891827 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 04:16:32.891827 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 04:16:32.891827 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 04:16:32.891827 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 04:16:32.891827 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 04:16:32.891827 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 04:16:32.891827 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 04:16:32.891827 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 04:16:32.892605 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 04:16:32.892605 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 04:16:32.892605 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 04:16:32.892605 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 04:16:32.892605 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 04:16:32.892605 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 04:16:32.892605 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 04:16:32.892605 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 04:16:32.892605 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 04:16:32.892605 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 04:16:32.892605 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 06 04:16:32.892605 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 04:16:32.892605 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 34/73] 10.222.0.30 () {56 vars in 1114 bytes} [Tue Sep 6 04:16:32 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 04:16:32.893817 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9dc3bb4-8da5-4fb1-a3ff-e546ddacd510 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:16:32.893817 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9dc3bb4-8da5-4fb1-a3ff-e546ddacd510 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:16:32.893817 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9dc3bb4-8da5-4fb1-a3ff-e546ddacd510 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:16:32.906995 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:16:32.913231 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-e9dc3bb4-8da5-4fb1-a3ff-e546ddacd510 None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:16:32.914473 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-e9dc3bb4-8da5-4fb1-a3ff-e546ddacd510 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:16:32.915279 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6edaae-f6c3-4996-a101-b46f344e4afb None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:16:32.948859 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975d8074-bf1f-4057-a78c-79f032420e21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:16:32.949499 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975d8074-bf1f-4057-a78c-79f032420e21 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:16:32.949778 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975d8074-bf1f-4057-a78c-79f032420e21 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:16:32.950051 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975d8074-bf1f-4057-a78c-79f032420e21 None admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:16:32.950718 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975d8074-bf1f-4057-a78c-79f032420e21 None admin] RBAC: Adding query filter params (name=member) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:16:32.951019 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975d8074-bf1f-4057-a78c-79f032420e21 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:16:32.952412 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975d8074-bf1f-4057-a78c-79f032420e21 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:16:32.961043 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 40/74] 10.222.0.30 () {56 vars in 1101 bytes} [Tue Sep 6 04:16:32 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:16:32.962999 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975d8074-bf1f-4057-a78c-79f032420e21 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:16:32.982717 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 35/75] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Sep 6 04:16:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 04:16:32.983564 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9dc3bb4-8da5-4fb1-a3ff-e546ddacd510 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:16:33.078519 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975d8074-bf1f-4057-a78c-79f032420e21 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:16:33.093327 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9dc3bb4-8da5-4fb1-a3ff-e546ddacd510 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:16:33.113234 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073a405c-2a34-445a-aa56-0daa0ef7b6c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:16:33.113896 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073a405c-2a34-445a-aa56-0daa0ef7b6c8 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:16:33.114202 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073a405c-2a34-445a-aa56-0daa0ef7b6c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:16:33.114481 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073a405c-2a34-445a-aa56-0daa0ef7b6c8 None admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:16:33.115148 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073a405c-2a34-445a-aa56-0daa0ef7b6c8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:16:33.115451 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073a405c-2a34-445a-aa56-0daa0ef7b6c8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:16:33.117024 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073a405c-2a34-445a-aa56-0daa0ef7b6c8 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:16:33.118628 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:16:33.126961 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 41/76] 10.222.0.30 () {56 vars in 1099 bytes} [Tue Sep 6 04:16:32 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:16:33.128477 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073a405c-2a34-445a-aa56-0daa0ef7b6c8 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:16:33.134394 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300ffc69-3a86-4be1-aef6-ef3131fd4aba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:16:33.135025 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300ffc69-3a86-4be1-aef6-ef3131fd4aba None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:16:33.135309 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300ffc69-3a86-4be1-aef6-ef3131fd4aba None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:16:33.135570 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300ffc69-3a86-4be1-aef6-ef3131fd4aba None admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:16:33.136242 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300ffc69-3a86-4be1-aef6-ef3131fd4aba None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:16:33.136543 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300ffc69-3a86-4be1-aef6-ef3131fd4aba None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:16:33.138019 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300ffc69-3a86-4be1-aef6-ef3131fd4aba None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:16:33.140831 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:16:33.148455 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 36/77] 10.222.0.30 () {56 vars in 1111 bytes} [Tue Sep 6 04:16:32 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:16:33.149274 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300ffc69-3a86-4be1-aef6-ef3131fd4aba None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:16:33.240901 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073a405c-2a34-445a-aa56-0daa0ef7b6c8 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:16:33.258154 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300ffc69-3a86-4be1-aef6-ef3131fd4aba None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:16:33.277620 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e293d8b-2114-41c1-a38a-dd6c8aeebd20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:16:33.278253 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e293d8b-2114-41c1-a38a-dd6c8aeebd20 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:16:33.278531 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e293d8b-2114-41c1-a38a-dd6c8aeebd20 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:16:33.278801 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e293d8b-2114-41c1-a38a-dd6c8aeebd20 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:16:33.288884 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e293d8b-2114-41c1-a38a-dd6c8aeebd20 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 04:16:33.293749 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-6e293d8b-2114-41c1-a38a-dd6c8aeebd20 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 06 04:16:33.293749 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 04:16:33.293749 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 04:16:33.293749 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 04:16:33.293749 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 04:16:33.293749 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 04:16:33.293749 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 04:16:33.293749 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 04:16:33.293749 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 04:16:33.293749 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 04:16:33.293749 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 04:16:33.293749 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 04:16:33.293749 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 04:16:33.293749 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 04:16:33.293749 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 04:16:33.293749 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 04:16:33.294583 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 04:16:33.294583 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 04:16:33.294583 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 04:16:33.294583 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 04:16:33.294583 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 04:16:33.294583 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 04:16:33.294583 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 04:16:33.294583 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 04:16:33.294583 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 04:16:33.294583 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 04:16:33.294583 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 04:16:33.294583 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 04:16:33.294583 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 04:16:33.294583 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 04:16:33.294583 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 04:16:33.294583 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 04:16:33.295365 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 04:16:33.295365 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 04:16:33.295365 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 04:16:33.295365 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 04:16:33.295365 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 04:16:33.295365 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 04:16:33.295365 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 04:16:33.295365 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 04:16:33.295365 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 04:16:33.295365 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 04:16:33.295365 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 06 04:16:33.295365 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 04:16:33.295943 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202589f2-cb0f-4a31-add7-e31be5d0e84a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:16:33.295943 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202589f2-cb0f-4a31-add7-e31be5d0e84a None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:16:33.296076 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 42/78] 10.222.0.30 () {56 vars in 1099 bytes} [Tue Sep 6 04:16:33 2022] GET /identity/v3/roles/member => generated 84 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 04:16:33.296147 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202589f2-cb0f-4a31-add7-e31be5d0e84a None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:16:33.296147 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202589f2-cb0f-4a31-add7-e31be5d0e84a None admin] PATH_INFO: `/v3/users/46380000cf7c43d197c6d8207cc91e2a` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:16:33.296269 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e293d8b-2114-41c1-a38a-dd6c8aeebd20 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:16:33.313231 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202589f2-cb0f-4a31-add7-e31be5d0e84a None admin] RBAC: Authorizing `identity:get_user(user_id=46380000cf7c43d197c6d8207cc91e2a)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:16:33.314736 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202589f2-cb0f-4a31-add7-e31be5d0e84a None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:16:33.316396 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 37/79] 10.222.0.30 () {56 vars in 1177 bytes} [Tue Sep 6 04:16:33 2022] GET /identity/v3/users/46380000cf7c43d197c6d8207cc91e2a => generated 275 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:16:33.316396 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202589f2-cb0f-4a31-add7-e31be5d0e84a None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:16:33.356567 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:16:33.383408 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:16:33.408158 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e293d8b-2114-41c1-a38a-dd6c8aeebd20 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:16:33.429141 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202589f2-cb0f-4a31-add7-e31be5d0e84a None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:16:33.451361 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4eac4a-b700-4dd8-972a-af25fa704e43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:16:33.452028 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4eac4a-b700-4dd8-972a-af25fa704e43 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:16:33.452310 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4eac4a-b700-4dd8-972a-af25fa704e43 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:16:33.452583 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4eac4a-b700-4dd8-972a-af25fa704e43 None admin] PATH_INFO: `/v3/users/b7c5062264774b85bf945fb9ac7602db` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:16:33.453737 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4eac4a-b700-4dd8-972a-af25fa704e43 None admin] RBAC: Authorizing `identity:get_user(user_id=b7c5062264774b85bf945fb9ac7602db)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:16:33.455181 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4eac4a-b700-4dd8-972a-af25fa704e43 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:16:33.456776 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 43/80] 10.222.0.30 () {56 vars in 1177 bytes} [Tue Sep 6 04:16:33 2022] GET /identity/v3/users/b7c5062264774b85bf945fb9ac7602db => generated 247 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:16:33.457560 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4eac4a-b700-4dd8-972a-af25fa704e43 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:16:33.472164 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c09a585-b384-4cd9-97e3-8cc930767d0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:16:33.472892 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c09a585-b384-4cd9-97e3-8cc930767d0f None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:16:33.473179 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c09a585-b384-4cd9-97e3-8cc930767d0f None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:16:33.473630 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c09a585-b384-4cd9-97e3-8cc930767d0f None admin] PATH_INFO: `/v3/users/46380000cf7c43d197c6d8207cc91e2a` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:16:33.474782 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c09a585-b384-4cd9-97e3-8cc930767d0f None admin] RBAC: Authorizing `identity:get_user(user_id=46380000cf7c43d197c6d8207cc91e2a)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:16:33.476376 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c09a585-b384-4cd9-97e3-8cc930767d0f None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:16:33.478009 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 38/81] 10.222.0.30 () {56 vars in 1177 bytes} [Tue Sep 6 04:16:33 2022] GET /identity/v3/users/46380000cf7c43d197c6d8207cc91e2a => generated 275 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:16:33.478781 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c09a585-b384-4cd9-97e3-8cc930767d0f None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:16:33.560800 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4eac4a-b700-4dd8-972a-af25fa704e43 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:16:33.583020 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c09a585-b384-4cd9-97e3-8cc930767d0f None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:16:33.584210 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:16:33.594832 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60580628-1ba0-432a-90cb-58f87994640d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:16:33.595475 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60580628-1ba0-432a-90cb-58f87994640d None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:16:33.595760 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60580628-1ba0-432a-90cb-58f87994640d None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:16:33.596036 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60580628-1ba0-432a-90cb-58f87994640d None admin] PATH_INFO: `/v3/domains/default` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:16:33.597162 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60580628-1ba0-432a-90cb-58f87994640d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:16:33.598658 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60580628-1ba0-432a-90cb-58f87994640d None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:16:33.600250 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 44/82] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Sep 6 04:16:33 2022] GET /identity/v3/domains/default => generated 199 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:16:33.601120 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60580628-1ba0-432a-90cb-58f87994640d None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:16:33.608072 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:16:33.618257 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8bebc4-29a1-49df-ad78-d667b345de49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:16:33.618880 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8bebc4-29a1-49df-ad78-d667b345de49 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:16:33.619161 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8bebc4-29a1-49df-ad78-d667b345de49 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:16:33.619490 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8bebc4-29a1-49df-ad78-d667b345de49 None admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:16:33.620160 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8bebc4-29a1-49df-ad78-d667b345de49 None admin] RBAC: Adding query filter params (name=member) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:16:33.620465 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8bebc4-29a1-49df-ad78-d667b345de49 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:16:33.621887 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8bebc4-29a1-49df-ad78-d667b345de49 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:16:33.630529 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 39/83] 10.222.0.30 () {56 vars in 1101 bytes} [Tue Sep 6 04:16:33 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:16:33.631965 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8bebc4-29a1-49df-ad78-d667b345de49 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:16:33.704887 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60580628-1ba0-432a-90cb-58f87994640d None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:16:33.729596 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8bebc4-29a1-49df-ad78-d667b345de49 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:16:33.739041 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8f0ba8-a7ba-4530-8707-1c1388bd7d4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:16:33.739805 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8f0ba8-a7ba-4530-8707-1c1388bd7d4c None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:16:33.740092 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8f0ba8-a7ba-4530-8707-1c1388bd7d4c None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:16:33.740372 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8f0ba8-a7ba-4530-8707-1c1388bd7d4c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:16:33.770127 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d21ef27-76d7-433b-b8db-1e1a29987b50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:16:33.770761 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d21ef27-76d7-433b-b8db-1e1a29987b50 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:16:33.771040 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d21ef27-76d7-433b-b8db-1e1a29987b50 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:16:33.771319 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d21ef27-76d7-433b-b8db-1e1a29987b50 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:16:33.800817 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:16:33.810653 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a8f0ba8-a7ba-4530-8707-1c1388bd7d4c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:16:33.813984 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a8f0ba8-a7ba-4530-8707-1c1388bd7d4c None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:16:33.826212 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:16:33.839705 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d21ef27-76d7-433b-b8db-1e1a29987b50 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:16:33.841200 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d21ef27-76d7-433b-b8db-1e1a29987b50 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:16:33.910878 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 45/84] 10.222.0.30 () {58 vars in 1282 bytes} [Tue Sep 6 04:16:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 06 04:16:33.911643 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8f0ba8-a7ba-4530-8707-1c1388bd7d4c None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:16:33.940548 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 40/85] 10.222.0.30 () {58 vars in 1282 bytes} [Tue Sep 6 04:16:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 06 04:16:33.941389 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d21ef27-76d7-433b-b8db-1e1a29987b50 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:16:34.001674 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:16:34.011352 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8f0ba8-a7ba-4530-8707-1c1388bd7d4c None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:16:34.027685 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:16:34.042862 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d21ef27-76d7-433b-b8db-1e1a29987b50 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:16:34.045071 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba5ca6d-a334-4a16-bc7c-b00cd490c117 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:16:34.045716 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba5ca6d-a334-4a16-bc7c-b00cd490c117 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:16:34.046000 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba5ca6d-a334-4a16-bc7c-b00cd490c117 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:16:34.046274 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba5ca6d-a334-4a16-bc7c-b00cd490c117 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:16:34.084303 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bda5b19-0eaf-404e-9658-be592544295d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:16:34.085610 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bda5b19-0eaf-404e-9658-be592544295d None admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:16:34.085903 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bda5b19-0eaf-404e-9658-be592544295d None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:16:34.086186 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bda5b19-0eaf-404e-9658-be592544295d None admin] PATH_INFO: `/v3/users` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:16:34.086857 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bda5b19-0eaf-404e-9658-be592544295d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:16:34.088319 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bda5b19-0eaf-404e-9658-be592544295d None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:16:34.125448 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba5ca6d-a334-4a16-bc7c-b00cd490c117 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:16:34.126933 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba5ca6d-a334-4a16-bc7c-b00cd490c117 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:16:34.215183 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:16:34.224627 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 46/86] 10.222.0.30 () {58 vars in 1282 bytes} [Tue Sep 6 04:16:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 06 04:16:34.225616 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba5ca6d-a334-4a16-bc7c-b00cd490c117 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:16:34.230090 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 41/87] 10.222.0.30 () {60 vars in 1132 bytes} [Tue Sep 6 04:16:33 2022] POST /identity/v3/users => generated 289 bytes in 288 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 04:16:34.230522 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bda5b19-0eaf-404e-9658-be592544295d None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:16:34.343542 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba5ca6d-a334-4a16-bc7c-b00cd490c117 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:16:34.361910 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bda5b19-0eaf-404e-9658-be592544295d None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:16:34.395610 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca8920a-e93c-496a-a8e7-c3ca33b7e8cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:16:34.396254 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca8920a-e93c-496a-a8e7-c3ca33b7e8cb None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:16:34.396530 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca8920a-e93c-496a-a8e7-c3ca33b7e8cb None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:16:34.397121 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca8920a-e93c-496a-a8e7-c3ca33b7e8cb None admin] PATH_INFO: `/v3/users/46380000cf7c43d197c6d8207cc91e2a` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:16:34.400559 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca8920a-e93c-496a-a8e7-c3ca33b7e8cb None admin] RBAC: Authorizing `identity:get_user(user_id=46380000cf7c43d197c6d8207cc91e2a)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:16:34.402105 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca8920a-e93c-496a-a8e7-c3ca33b7e8cb None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:16:34.404589 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 47/88] 10.222.0.30 () {56 vars in 1177 bytes} [Tue Sep 6 04:16:34 2022] GET /identity/v3/users/46380000cf7c43d197c6d8207cc91e2a => generated 275 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:16:34.410221 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca8920a-e93c-496a-a8e7-c3ca33b7e8cb None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:16:34.416841 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8020388-065a-4f21-bf59-b98def4beec9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:16:34.417690 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8020388-065a-4f21-bf59-b98def4beec9 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:16:34.422498 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8020388-065a-4f21-bf59-b98def4beec9 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:16:34.422498 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8020388-065a-4f21-bf59-b98def4beec9 None admin] PATH_INFO: `/v3/projects/a88c6977d0554a1eab3e30118ede2f46` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:16:34.422498 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8020388-065a-4f21-bf59-b98def4beec9 None admin] RBAC: Authorizing `identity:get_project(project_id=a88c6977d0554a1eab3e30118ede2f46)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:16:34.422793 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8020388-065a-4f21-bf59-b98def4beec9 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:16:34.426078 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 42/89] 10.222.0.30 () {56 vars in 1186 bytes} [Tue Sep 6 04:16:34 2022] GET /identity/v3/projects/a88c6977d0554a1eab3e30118ede2f46 => generated 298 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:16:34.426958 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8020388-065a-4f21-bf59-b98def4beec9 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:16:34.442539 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:16:34.485083 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:16:34.539017 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca8920a-e93c-496a-a8e7-c3ca33b7e8cb None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:16:34.562582 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8020388-065a-4f21-bf59-b98def4beec9 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:16:34.582088 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad1b427-3500-48c9-bfcb-e0d0961a522d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:16:34.583188 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad1b427-3500-48c9-bfcb-e0d0961a522d None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:16:34.583477 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad1b427-3500-48c9-bfcb-e0d0961a522d None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:16:34.583752 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad1b427-3500-48c9-bfcb-e0d0961a522d None admin] PATH_INFO: `/v3/projects/a88c6977d0554a1eab3e30118ede2f46` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:16:34.584934 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad1b427-3500-48c9-bfcb-e0d0961a522d None admin] RBAC: Authorizing `identity:get_project(project_id=a88c6977d0554a1eab3e30118ede2f46)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:16:34.586644 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad1b427-3500-48c9-bfcb-e0d0961a522d None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:16:34.589527 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 48/90] 10.222.0.30 () {56 vars in 1186 bytes} [Tue Sep 6 04:16:34 2022] GET /identity/v3/projects/a88c6977d0554a1eab3e30118ede2f46 => generated 298 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:16:34.590394 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad1b427-3500-48c9-bfcb-e0d0961a522d None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:16:34.607195 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d823f31-9756-46e8-9cf7-0cc97aba4f40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:16:34.607845 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d823f31-9756-46e8-9cf7-0cc97aba4f40 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:16:34.608129 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d823f31-9756-46e8-9cf7-0cc97aba4f40 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:16:34.608507 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d823f31-9756-46e8-9cf7-0cc97aba4f40 None admin] PATH_INFO: `/v3/projects/a88c6977d0554a1eab3e30118ede2f46` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:16:34.609794 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d823f31-9756-46e8-9cf7-0cc97aba4f40 None admin] RBAC: Authorizing `identity:get_project(project_id=a88c6977d0554a1eab3e30118ede2f46)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:16:34.611254 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d823f31-9756-46e8-9cf7-0cc97aba4f40 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:16:34.613477 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 43/91] 10.222.0.30 () {56 vars in 1186 bytes} [Tue Sep 6 04:16:34 2022] GET /identity/v3/projects/a88c6977d0554a1eab3e30118ede2f46 => generated 298 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:16:34.614263 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d823f31-9756-46e8-9cf7-0cc97aba4f40 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:16:34.711658 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:16:34.736420 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad1b427-3500-48c9-bfcb-e0d0961a522d None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:16:34.736716 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d823f31-9756-46e8-9cf7-0cc97aba4f40 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:16:34.773590 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:16:34.775935 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9310a42-0049-4517-94ea-a651766bf12b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:16:34.776739 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9310a42-0049-4517-94ea-a651766bf12b None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:16:34.777164 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9310a42-0049-4517-94ea-a651766bf12b None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:16:34.777627 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9310a42-0049-4517-94ea-a651766bf12b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:16:34.778503 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9310a42-0049-4517-94ea-a651766bf12b None admin] RBAC: Adding query filter params (role.id=29607b2de39f40d68bb6ceef64b8a732, scope.project.id=a88c6977d0554a1eab3e30118ede2f46, user.id=46380000cf7c43d197c6d8207cc91e2a) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:16:34.778957 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9310a42-0049-4517-94ea-a651766bf12b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:16:34.780670 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9310a42-0049-4517-94ea-a651766bf12b None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:16:34.782299 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9217ae-1bbe-4e97-9353-34efbaadcdac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:16:34.782978 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9217ae-1bbe-4e97-9353-34efbaadcdac None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:16:34.783262 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9217ae-1bbe-4e97-9353-34efbaadcdac None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:16:34.787072 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9217ae-1bbe-4e97-9353-34efbaadcdac None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:16:34.791517 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 44/92] 10.222.0.30 () {56 vars in 1374 bytes} [Tue Sep 6 04:16:34 2022] GET /identity/v3/role_assignments?user.id=46380000cf7c43d197c6d8207cc91e2a&scope.project.id=a88c6977d0554a1eab3e30118ede2f46&role.id=29607b2de39f40d68bb6ceef64b8a732 => generated 259 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:16:34.792657 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9310a42-0049-4517-94ea-a651766bf12b None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:16:34.875536 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9217ae-1bbe-4e97-9353-34efbaadcdac None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:16:34.877742 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9217ae-1bbe-4e97-9353-34efbaadcdac None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:16:34.937929 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9310a42-0049-4517-94ea-a651766bf12b None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:16:34.974624 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4579e26-fc5d-4912-ac0c-3b88acaa792c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:16:34.976102 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4579e26-fc5d-4912-ac0c-3b88acaa792c None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:16:34.976412 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4579e26-fc5d-4912-ac0c-3b88acaa792c None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:16:34.976695 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4579e26-fc5d-4912-ac0c-3b88acaa792c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:16:34.977478 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4579e26-fc5d-4912-ac0c-3b88acaa792c None admin] RBAC: Adding query filter params (role.id=ea4c814713b149108e488de855b025e4, scope.project.id=a88c6977d0554a1eab3e30118ede2f46, user.id=b7c5062264774b85bf945fb9ac7602db) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:16:34.977790 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4579e26-fc5d-4912-ac0c-3b88acaa792c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:16:34.979262 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4579e26-fc5d-4912-ac0c-3b88acaa792c None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:16:34.985713 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 45/93] 10.222.0.30 () {56 vars in 1374 bytes} [Tue Sep 6 04:16:34 2022] GET /identity/v3/role_assignments?user.id=b7c5062264774b85bf945fb9ac7602db&scope.project.id=a88c6977d0554a1eab3e30118ede2f46&role.id=ea4c814713b149108e488de855b025e4 => generated 259 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:16:34.986741 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4579e26-fc5d-4912-ac0c-3b88acaa792c None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:16:34.996761 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:16:35.072134 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 49/94] 10.222.0.30 () {58 vars in 1282 bytes} [Tue Sep 6 04:16:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 482 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 06 04:16:35.078747 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9217ae-1bbe-4e97-9353-34efbaadcdac None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:16:35.137721 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:16:35.182020 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4579e26-fc5d-4912-ac0c-3b88acaa792c None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:16:35.254309 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9217ae-1bbe-4e97-9353-34efbaadcdac None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:16:35.257128 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd29a77a-8536-4c3b-804f-f3d787afb729 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:16:35.257770 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd29a77a-8536-4c3b-804f-f3d787afb729 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:16:35.258049 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd29a77a-8536-4c3b-804f-f3d787afb729 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:16:35.258322 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd29a77a-8536-4c3b-804f-f3d787afb729 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:16:35.259057 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd29a77a-8536-4c3b-804f-f3d787afb729 None admin] RBAC: Adding query filter params (role.id=77197446dd2541b9a4fab9a654c08fc9, scope.project.id=a88c6977d0554a1eab3e30118ede2f46, user.id=46380000cf7c43d197c6d8207cc91e2a) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:16:35.259365 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd29a77a-8536-4c3b-804f-f3d787afb729 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:16:35.260812 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd29a77a-8536-4c3b-804f-f3d787afb729 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:16:35.273767 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 46/95] 10.222.0.30 () {56 vars in 1374 bytes} [Tue Sep 6 04:16:34 2022] GET /identity/v3/role_assignments?user.id=46380000cf7c43d197c6d8207cc91e2a&scope.project.id=a88c6977d0554a1eab3e30118ede2f46&role.id=77197446dd2541b9a4fab9a654c08fc9 => generated 259 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:16:35.300575 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eeafb3f-cfd8-4cdf-9b62-aa4ba4c9f6d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:16:35.301226 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eeafb3f-cfd8-4cdf-9b62-aa4ba4c9f6d3 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:16:35.301531 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eeafb3f-cfd8-4cdf-9b62-aa4ba4c9f6d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:16:35.301802 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eeafb3f-cfd8-4cdf-9b62-aa4ba4c9f6d3 None admin] PATH_INFO: `/v3/projects/194c60f0a143489a81851796cf870320` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:16:35.302945 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eeafb3f-cfd8-4cdf-9b62-aa4ba4c9f6d3 None admin] RBAC: Authorizing `identity:get_project(project_id=194c60f0a143489a81851796cf870320)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:16:35.305190 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eeafb3f-cfd8-4cdf-9b62-aa4ba4c9f6d3 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:16:35.306962 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 50/96] 10.222.0.30 () {56 vars in 1186 bytes} [Tue Sep 6 04:16:35 2022] GET /identity/v3/projects/194c60f0a143489a81851796cf870320 => generated 312 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 04:16:35.312903 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd29a77a-8536-4c3b-804f-f3d787afb729 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:16:35.428764 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:16:35.505426 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd29a77a-8536-4c3b-804f-f3d787afb729 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:16:35.541889 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20356d74-5583-4d2e-ac1b-ec2138c20058 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:16:35.542515 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20356d74-5583-4d2e-ac1b-ec2138c20058 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:16:35.542796 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20356d74-5583-4d2e-ac1b-ec2138c20058 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:16:35.543070 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20356d74-5583-4d2e-ac1b-ec2138c20058 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:16:35.543808 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20356d74-5583-4d2e-ac1b-ec2138c20058 None admin] RBAC: Adding query filter params (role.id=29607b2de39f40d68bb6ceef64b8a732, scope.project.id=194c60f0a143489a81851796cf870320, user.id=46380000cf7c43d197c6d8207cc91e2a) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:16:35.544114 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20356d74-5583-4d2e-ac1b-ec2138c20058 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:16:35.545596 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20356d74-5583-4d2e-ac1b-ec2138c20058 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:16:35.552345 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 47/97] 10.222.0.30 () {56 vars in 1374 bytes} [Tue Sep 6 04:16:35 2022] GET /identity/v3/role_assignments?user.id=46380000cf7c43d197c6d8207cc91e2a&scope.project.id=194c60f0a143489a81851796cf870320&role.id=29607b2de39f40d68bb6ceef64b8a732 => generated 259 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:16:36.766383 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89afd777-4210-4396-899d-91c17f6731ef None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:16:36.766884 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89afd777-4210-4396-899d-91c17f6731ef None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:16:36.766884 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89afd777-4210-4396-899d-91c17f6731ef None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:16:36.768472 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 51/98] 10.222.0.30 () {54 vars in 927 bytes} [Tue Sep 6 04:16:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 04:16:36.773887 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31dc62a7-9f81-4cc7-bd8e-a8b891cdc260 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:16:36.774301 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31dc62a7-9f81-4cc7-bd8e-a8b891cdc260 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:16:36.774462 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31dc62a7-9f81-4cc7-bd8e-a8b891cdc260 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:16:36.802289 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-31dc62a7-9f81-4cc7-bd8e-a8b891cdc260 None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:16:36.804301 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-31dc62a7-9f81-4cc7-bd8e-a8b891cdc260 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:16:36.888017 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd70d54-c8fe-4bd4-8854-de07df321b62 None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:16:36.888330 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd70d54-c8fe-4bd4-8854-de07df321b62 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:16:36.888586 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd70d54-c8fe-4bd4-8854-de07df321b62 None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:16:36.890581 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 52/99] 10.222.0.30 () {54 vars in 927 bytes} [Tue Sep 6 04:16:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 04:16:36.903125 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 48/100] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Sep 6 04:16:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 04:16:36.907120 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae96f5f6-1b67-4d49-a77f-21be36f925eb None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:16:36.907409 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae96f5f6-1b67-4d49-a77f-21be36f925eb None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:16:36.907684 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae96f5f6-1b67-4d49-a77f-21be36f925eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:16:36.925558 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-ae96f5f6-1b67-4d49-a77f-21be36f925eb None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:16:36.926745 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-ae96f5f6-1b67-4d49-a77f-21be36f925eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:16:37.015896 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 53/101] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Sep 6 04:16:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 04:16:37.290774 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31dc62a7-9f81-4cc7-bd8e-a8b891cdc260 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:16:37.403708 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:16:37.505551 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31dc62a7-9f81-4cc7-bd8e-a8b891cdc260 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:16:37.561689 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0044545d-9d5d-4b8c-930b-261b3650f090 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:16:37.562344 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0044545d-9d5d-4b8c-930b-261b3650f090 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:16:37.562627 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0044545d-9d5d-4b8c-930b-261b3650f090 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:16:37.562906 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0044545d-9d5d-4b8c-930b-261b3650f090 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:16:37.590325 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0044545d-9d5d-4b8c-930b-261b3650f090 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 04:16:37.595260 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-0044545d-9d5d-4b8c-930b-261b3650f090 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 06 04:16:37.595260 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 04:16:37.595260 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 04:16:37.595260 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 04:16:37.595260 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 04:16:37.595260 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 04:16:37.595260 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 04:16:37.595260 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 04:16:37.595260 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 04:16:37.595260 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 04:16:37.595260 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 04:16:37.595260 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 04:16:37.595260 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 04:16:37.595260 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 04:16:37.595260 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 04:16:37.595260 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 04:16:37.596116 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 04:16:37.596116 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 04:16:37.596116 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 04:16:37.596116 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 04:16:37.596116 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 04:16:37.596116 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 04:16:37.596116 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 04:16:37.596116 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 04:16:37.596116 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 04:16:37.596116 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 04:16:37.596116 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 04:16:37.596116 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 04:16:37.596116 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 04:16:37.596116 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 04:16:37.596116 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 04:16:37.596116 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 04:16:37.597352 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 04:16:37.597352 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 04:16:37.597352 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 04:16:37.597352 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 04:16:37.597352 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 04:16:37.597352 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 04:16:37.597352 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 04:16:37.597352 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 04:16:37.597352 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 04:16:37.597352 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 04:16:37.597352 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 06 04:16:37.597352 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 04:16:37.597352 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 49/102] 10.222.0.30 () {56 vars in 1099 bytes} [Tue Sep 6 04:16:37 2022] GET /identity/v3/roles/reader => generated 84 bytes in 307 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 04:16:37.608375 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae96f5f6-1b67-4d49-a77f-21be36f925eb None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:16:37.652942 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0044545d-9d5d-4b8c-930b-261b3650f090 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:16:37.655158 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:16:37.821402 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0044545d-9d5d-4b8c-930b-261b3650f090 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:16:37.851241 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:16:37.854175 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae96f5f6-1b67-4d49-a77f-21be36f925eb None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:16:37.872514 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22669077-59f9-4766-b754-76c65546f58b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:16:37.873428 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22669077-59f9-4766-b754-76c65546f58b None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:16:37.873847 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22669077-59f9-4766-b754-76c65546f58b None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:16:37.874338 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22669077-59f9-4766-b754-76c65546f58b None admin] PATH_INFO: `/v3/domains/default` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:16:37.875710 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22669077-59f9-4766-b754-76c65546f58b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:16:37.877366 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22669077-59f9-4766-b754-76c65546f58b None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:16:37.879804 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 50/103] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Sep 6 04:16:37 2022] GET /identity/v3/domains/default => generated 199 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 04:16:37.881684 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31dd90c5-213c-466a-bea0-27c1c35342f4 None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:16:37.882514 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31dd90c5-213c-466a-bea0-27c1c35342f4 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:16:37.882982 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31dd90c5-213c-466a-bea0-27c1c35342f4 None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:16:37.884647 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 51/104] 10.222.0.30 () {54 vars in 927 bytes} [Tue Sep 6 04:16:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 04:16:37.893356 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31dd90c5-213c-466a-bea0-27c1c35342f4 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:16:37.934904 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14d9856-913b-4354-bad5-30bbbfa3db54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:16:37.935537 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14d9856-913b-4354-bad5-30bbbfa3db54 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:16:37.935819 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14d9856-913b-4354-bad5-30bbbfa3db54 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:16:37.936098 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14d9856-913b-4354-bad5-30bbbfa3db54 None admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:16:37.936785 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14d9856-913b-4354-bad5-30bbbfa3db54 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:16:37.937092 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14d9856-913b-4354-bad5-30bbbfa3db54 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:16:37.938529 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14d9856-913b-4354-bad5-30bbbfa3db54 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:16:37.960514 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 54/105] 10.222.0.30 () {56 vars in 1101 bytes} [Tue Sep 6 04:16:37 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 353 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:16:37.962107 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7224eb5-9ae3-45d8-97e4-c568c117231c None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:16:37.962514 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7224eb5-9ae3-45d8-97e4-c568c117231c None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:16:37.962798 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7224eb5-9ae3-45d8-97e4-c568c117231c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:16:38.000231 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-d7224eb5-9ae3-45d8-97e4-c568c117231c None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:16:38.001451 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-d7224eb5-9ae3-45d8-97e4-c568c117231c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:16:38.098039 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31dd90c5-213c-466a-bea0-27c1c35342f4 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:16:38.110170 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 55/106] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Sep 6 04:16:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 04:16:38.116995 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7224eb5-9ae3-45d8-97e4-c568c117231c None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:16:38.143090 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:16:38.167308 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da08fd12-c190-4788-a5cf-6579b6149729 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:16:38.167984 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da08fd12-c190-4788-a5cf-6579b6149729 None admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:16:38.168265 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da08fd12-c190-4788-a5cf-6579b6149729 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:16:38.168539 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da08fd12-c190-4788-a5cf-6579b6149729 None admin] PATH_INFO: `/v3/projects` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:16:38.169205 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da08fd12-c190-4788-a5cf-6579b6149729 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:16:38.170686 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da08fd12-c190-4788-a5cf-6579b6149729 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:16:38.259116 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7224eb5-9ae3-45d8-97e4-c568c117231c None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:16:38.295995 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a60258-5bfc-4167-841d-a52ec061a94d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:16:38.296633 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a60258-5bfc-4167-841d-a52ec061a94d None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:16:38.296914 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a60258-5bfc-4167-841d-a52ec061a94d None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:16:38.297233 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a60258-5bfc-4167-841d-a52ec061a94d None admin] PATH_INFO: `/v3/users/501af79e7c0d4d768256c193a0305538` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:16:38.308640 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 52/107] 10.222.0.30 () {60 vars in 1141 bytes} [Tue Sep 6 04:16:37 2022] POST /identity/v3/projects => generated 302 bytes in 422 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 04:16:38.309953 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb51624d-a2a4-4673-8b41-3d5bea66d9a4 None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:16:38.310242 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb51624d-a2a4-4673-8b41-3d5bea66d9a4 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:16:38.310517 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb51624d-a2a4-4673-8b41-3d5bea66d9a4 None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:16:38.314413 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 53/108] 10.222.0.30 () {54 vars in 927 bytes} [Tue Sep 6 04:16:38 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 04:16:38.323005 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ce4dba-30e5-441d-9591-4720b9f59a51 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:16:38.327919 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ce4dba-30e5-441d-9591-4720b9f59a51 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:16:38.327919 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ce4dba-30e5-441d-9591-4720b9f59a51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:16:38.344464 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a60258-5bfc-4167-841d-a52ec061a94d None admin] RBAC: Authorizing `identity:get_user(user_id=501af79e7c0d4d768256c193a0305538)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:16:38.350600 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a60258-5bfc-4167-841d-a52ec061a94d None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:16:38.360095 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 56/109] 10.222.0.30 () {56 vars in 1177 bytes} [Tue Sep 6 04:16:38 2022] GET /identity/v3/users/501af79e7c0d4d768256c193a0305538 => generated 289 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:16:38.366189 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a60258-5bfc-4167-841d-a52ec061a94d None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:16:38.372146 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-d1ce4dba-30e5-441d-9591-4720b9f59a51 None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:16:38.373413 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-d1ce4dba-30e5-441d-9591-4720b9f59a51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:16:38.439747 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:16:38.470738 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 54/110] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Sep 6 04:16:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 04:16:38.471923 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ce4dba-30e5-441d-9591-4720b9f59a51 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:16:38.502761 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:16:38.550271 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a60258-5bfc-4167-841d-a52ec061a94d None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:16:38.591044 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af4b7ad-de33-45b4-a67b-dfd067e666ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:16:38.591898 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af4b7ad-de33-45b4-a67b-dfd067e666ee None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:16:38.592353 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af4b7ad-de33-45b4-a67b-dfd067e666ee None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:16:38.592771 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af4b7ad-de33-45b4-a67b-dfd067e666ee None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:16:38.640580 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ce4dba-30e5-441d-9591-4720b9f59a51 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:16:38.672402 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af4b7ad-de33-45b4-a67b-dfd067e666ee None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:16:38.674189 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af4b7ad-de33-45b4-a67b-dfd067e666ee None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:16:38.691438 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d913bd7-66f6-40e7-8f26-effe039dbb78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:16:38.692545 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d913bd7-66f6-40e7-8f26-effe039dbb78 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:16:38.692831 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d913bd7-66f6-40e7-8f26-effe039dbb78 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:16:38.693213 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d913bd7-66f6-40e7-8f26-effe039dbb78 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:16:38.707709 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d913bd7-66f6-40e7-8f26-effe039dbb78 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 04:16:38.714909 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-1d913bd7-66f6-40e7-8f26-effe039dbb78 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 06 04:16:38.714909 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 04:16:38.714909 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 04:16:38.714909 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 04:16:38.714909 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 04:16:38.714909 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 04:16:38.714909 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 04:16:38.714909 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 04:16:38.714909 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 04:16:38.714909 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 04:16:38.714909 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 04:16:38.714909 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 04:16:38.714909 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 04:16:38.714909 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 04:16:38.714909 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 04:16:38.714909 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 04:16:38.715938 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 04:16:38.715938 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 04:16:38.715938 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 04:16:38.715938 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 04:16:38.715938 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 04:16:38.715938 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 04:16:38.715938 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 04:16:38.715938 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 04:16:38.715938 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 04:16:38.715938 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 04:16:38.715938 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 04:16:38.715938 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 04:16:38.715938 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 04:16:38.715938 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 04:16:38.715938 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 04:16:38.715938 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 04:16:38.716891 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 04:16:38.716891 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 04:16:38.716891 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 04:16:38.716891 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 04:16:38.716891 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 04:16:38.716891 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 04:16:38.716891 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 04:16:38.716891 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 04:16:38.716891 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 04:16:38.716891 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 04:16:38.716891 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 06 04:16:38.716891 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 04:16:38.718701 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 55/111] 10.222.0.30 () {56 vars in 1099 bytes} [Tue Sep 6 04:16:38 2022] GET /identity/v3/roles/member => generated 84 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 04:16:38.721125 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b59e6d-7e61-4a42-91bf-03401e656559 None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:16:38.721591 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b59e6d-7e61-4a42-91bf-03401e656559 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:16:38.722509 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b59e6d-7e61-4a42-91bf-03401e656559 None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:16:38.724441 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 56/112] 10.222.0.30 () {54 vars in 927 bytes} [Tue Sep 6 04:16:38 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 04:16:38.726189 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b59e6d-7e61-4a42-91bf-03401e656559 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:16:38.769567 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:16:38.774633 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:16:38.802041 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 57/113] 10.222.0.30 () {58 vars in 1282 bytes} [Tue Sep 6 04:16:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 437 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 06 04:16:38.804582 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3074ec6-4f39-4136-9997-96984c3a8cc3 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:16:38.805051 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3074ec6-4f39-4136-9997-96984c3a8cc3 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:16:38.806091 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3074ec6-4f39-4136-9997-96984c3a8cc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:16:38.832429 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-d3074ec6-4f39-4136-9997-96984c3a8cc3 None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:16:38.833690 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-d3074ec6-4f39-4136-9997-96984c3a8cc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:16:38.870676 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b59e6d-7e61-4a42-91bf-03401e656559 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:16:38.911470 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 58/114] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Sep 6 04:16:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 04:16:38.912014 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fb01f1-6490-4411-b628-e4e9a26c4c2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:16:38.912014 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fb01f1-6490-4411-b628-e4e9a26c4c2e None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:16:38.912014 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fb01f1-6490-4411-b628-e4e9a26c4c2e None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:16:38.912014 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fb01f1-6490-4411-b628-e4e9a26c4c2e None admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:16:38.913680 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3fb01f1-6490-4411-b628-e4e9a26c4c2e None admin] RBAC: Adding query filter params (name=member) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:16:38.913997 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3fb01f1-6490-4411-b628-e4e9a26c4c2e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:16:38.915431 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3fb01f1-6490-4411-b628-e4e9a26c4c2e None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:16:38.918212 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3074ec6-4f39-4136-9997-96984c3a8cc3 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:16:38.927509 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 57/115] 10.222.0.30 () {56 vars in 1101 bytes} [Tue Sep 6 04:16:38 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:16:38.929933 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fb01f1-6490-4411-b628-e4e9a26c4c2e None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:16:39.032474 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3074ec6-4f39-4136-9997-96984c3a8cc3 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:16:39.041441 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:16:39.047829 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:16:39.055958 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fb01f1-6490-4411-b628-e4e9a26c4c2e None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:16:39.073539 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd43e09-62ab-475b-92b5-2259f931f60d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:16:39.074188 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd43e09-62ab-475b-92b5-2259f931f60d None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:16:39.074518 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd43e09-62ab-475b-92b5-2259f931f60d None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:16:39.074804 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd43e09-62ab-475b-92b5-2259f931f60d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:16:39.086401 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd43e09-62ab-475b-92b5-2259f931f60d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 04:16:39.092103 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-9bd43e09-62ab-475b-92b5-2259f931f60d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 06 04:16:39.092103 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 04:16:39.092103 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 04:16:39.092103 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 04:16:39.092103 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 04:16:39.092103 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 04:16:39.092103 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 04:16:39.092103 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 04:16:39.092103 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 04:16:39.092103 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 04:16:39.092103 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 04:16:39.092103 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 04:16:39.092103 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 04:16:39.092103 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 04:16:39.092103 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 04:16:39.092103 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 04:16:39.092954 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 04:16:39.092954 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 04:16:39.092954 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 04:16:39.092954 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 04:16:39.092954 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 04:16:39.092954 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 04:16:39.092954 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 04:16:39.092954 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 04:16:39.092954 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 04:16:39.092954 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 04:16:39.092954 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 04:16:39.092954 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 04:16:39.092954 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 04:16:39.092954 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 04:16:39.092954 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 04:16:39.092954 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 04:16:39.093754 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 04:16:39.093754 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 04:16:39.093754 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 04:16:39.093754 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 04:16:39.093754 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 04:16:39.093754 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 04:16:39.093754 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 04:16:39.093754 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 04:16:39.093754 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 04:16:39.093754 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 04:16:39.093754 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 06 04:16:39.093754 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 04:16:39.093754 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 59/116] 10.222.0.30 () {56 vars in 1114 bytes} [Tue Sep 6 04:16:38 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 04:16:39.096787 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdcb9e6-6d22-4248-9334-7454e72e7acc None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:16:39.097137 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdcb9e6-6d22-4248-9334-7454e72e7acc None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:16:39.097562 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdcb9e6-6d22-4248-9334-7454e72e7acc None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:16:39.098554 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5411e4a8-0a82-4a23-9e73-8b0498f391c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:16:39.099558 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 60/117] 10.222.0.30 () {54 vars in 927 bytes} [Tue Sep 6 04:16:39 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 04:16:39.099781 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5411e4a8-0a82-4a23-9e73-8b0498f391c6 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:16:39.100543 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fdcb9e6-6d22-4248-9334-7454e72e7acc None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:16:39.101103 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5411e4a8-0a82-4a23-9e73-8b0498f391c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:16:39.101581 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5411e4a8-0a82-4a23-9e73-8b0498f391c6 None admin] PATH_INFO: `/v3/projects/a88c6977d0554a1eab3e30118ede2f46` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:16:39.102934 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5411e4a8-0a82-4a23-9e73-8b0498f391c6 None admin] RBAC: Authorizing `identity:get_project(project_id=a88c6977d0554a1eab3e30118ede2f46)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:16:39.104570 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5411e4a8-0a82-4a23-9e73-8b0498f391c6 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:16:39.107195 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 58/118] 10.222.0.30 () {56 vars in 1186 bytes} [Tue Sep 6 04:16:38 2022] GET /identity/v3/projects/a88c6977d0554a1eab3e30118ede2f46 => generated 298 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:16:39.108255 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5411e4a8-0a82-4a23-9e73-8b0498f391c6 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:16:39.218778 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fdcb9e6-6d22-4248-9334-7454e72e7acc None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:16:39.229377 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5411e4a8-0a82-4a23-9e73-8b0498f391c6 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:16:39.255727 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd539d4-168a-454e-8918-ec71d699a064 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:16:39.256386 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd539d4-168a-454e-8918-ec71d699a064 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:16:39.256663 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd539d4-168a-454e-8918-ec71d699a064 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:16:39.256945 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd539d4-168a-454e-8918-ec71d699a064 None admin] PATH_INFO: `/v3/users/46380000cf7c43d197c6d8207cc91e2a` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:16:39.258077 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd539d4-168a-454e-8918-ec71d699a064 None admin] RBAC: Authorizing `identity:get_user(user_id=46380000cf7c43d197c6d8207cc91e2a)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:16:39.259555 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd539d4-168a-454e-8918-ec71d699a064 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:16:39.265509 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 61/119] 10.222.0.30 () {56 vars in 1177 bytes} [Tue Sep 6 04:16:39 2022] GET /identity/v3/users/46380000cf7c43d197c6d8207cc91e2a => generated 275 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:16:39.265509 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5de7ea-5458-41a4-802a-e036e1635c7e None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:16:39.265509 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5de7ea-5458-41a4-802a-e036e1635c7e None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:16:39.265509 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5de7ea-5458-41a4-802a-e036e1635c7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:16:39.270565 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e066e469-8a14-4afc-bf7a-9a6d5cf0410f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:16:39.271288 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e066e469-8a14-4afc-bf7a-9a6d5cf0410f None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:16:39.271757 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e066e469-8a14-4afc-bf7a-9a6d5cf0410f None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:16:39.272305 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e066e469-8a14-4afc-bf7a-9a6d5cf0410f None admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:16:39.273260 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e066e469-8a14-4afc-bf7a-9a6d5cf0410f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:16:39.273644 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e066e469-8a14-4afc-bf7a-9a6d5cf0410f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:16:39.275008 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e066e469-8a14-4afc-bf7a-9a6d5cf0410f None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:16:39.281892 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-fd5de7ea-5458-41a4-802a-e036e1635c7e None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:16:39.283010 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-fd5de7ea-5458-41a4-802a-e036e1635c7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:16:39.284117 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 59/120] 10.222.0.30 () {56 vars in 1111 bytes} [Tue Sep 6 04:16:39 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:16:39.284812 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e066e469-8a14-4afc-bf7a-9a6d5cf0410f None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:16:39.312870 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:16:39.314698 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:16:39.365508 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 62/121] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Sep 6 04:16:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 04:16:39.366791 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5de7ea-5458-41a4-802a-e036e1635c7e None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:16:39.414472 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e066e469-8a14-4afc-bf7a-9a6d5cf0410f None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:16:39.486882 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f4cf61-8a5f-4da4-9ee8-9619a4c61e47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:16:39.487441 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f4cf61-8a5f-4da4-9ee8-9619a4c61e47 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:16:39.487721 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f4cf61-8a5f-4da4-9ee8-9619a4c61e47 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:16:39.487993 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f4cf61-8a5f-4da4-9ee8-9619a4c61e47 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:16:39.488784 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f4cf61-8a5f-4da4-9ee8-9619a4c61e47 None admin] RBAC: Adding query filter params (role.id=3c011524d1df425b902b8b32c1b485c3, scope.project.id=a88c6977d0554a1eab3e30118ede2f46, user.id=501af79e7c0d4d768256c193a0305538) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:16:39.489098 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f4cf61-8a5f-4da4-9ee8-9619a4c61e47 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:16:39.490286 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5de7ea-5458-41a4-802a-e036e1635c7e None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:16:39.490767 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f4cf61-8a5f-4da4-9ee8-9619a4c61e47 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:16:39.497361 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 60/122] 10.222.0.30 () {56 vars in 1374 bytes} [Tue Sep 6 04:16:39 2022] GET /identity/v3/role_assignments?user.id=501af79e7c0d4d768256c193a0305538&scope.project.id=a88c6977d0554a1eab3e30118ede2f46&role.id=3c011524d1df425b902b8b32c1b485c3 => generated 259 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:16:39.497498 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f4cf61-8a5f-4da4-9ee8-9619a4c61e47 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:16:39.567966 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c19be41-e6b4-4fb8-b86d-b4522da5fbc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:16:39.569493 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c19be41-e6b4-4fb8-b86d-b4522da5fbc2 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:16:39.569777 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c19be41-e6b4-4fb8-b86d-b4522da5fbc2 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:16:39.570148 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c19be41-e6b4-4fb8-b86d-b4522da5fbc2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:16:39.590121 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c19be41-e6b4-4fb8-b86d-b4522da5fbc2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 04:16:39.596274 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-1c19be41-e6b4-4fb8-b86d-b4522da5fbc2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 04:16:39.596274 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 04:16:39.596274 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 04:16:39.596274 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 04:16:39.596274 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 04:16:39.596274 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 04:16:39.596274 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 04:16:39.596274 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 04:16:39.596274 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 04:16:39.596274 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 04:16:39.596274 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 04:16:39.596274 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 04:16:39.596274 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 04:16:39.596274 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 04:16:39.596274 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 04:16:39.596274 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 04:16:39.597171 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 04:16:39.597171 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 04:16:39.597171 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 04:16:39.597171 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 04:16:39.597171 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 04:16:39.597171 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 04:16:39.597171 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 04:16:39.597171 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 04:16:39.597171 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 04:16:39.597171 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 04:16:39.597171 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 04:16:39.597171 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 04:16:39.597171 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 04:16:39.597171 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 04:16:39.597171 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 04:16:39.597171 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 04:16:39.598002 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 04:16:39.598002 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 04:16:39.598002 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 04:16:39.598002 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 04:16:39.598002 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 04:16:39.598002 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 04:16:39.598002 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 04:16:39.598002 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 04:16:39.598002 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 04:16:39.598002 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 04:16:39.598002 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 04:16:39.598002 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 04:16:39.598002 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 63/123] 10.222.0.30 () {56 vars in 1096 bytes} [Tue Sep 6 04:16:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 04:16:39.598629 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c19be41-e6b4-4fb8-b86d-b4522da5fbc2 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:16:39.661209 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:16:39.671504 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:16:39.682294 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f4cf61-8a5f-4da4-9ee8-9619a4c61e47 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:16:39.727192 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932a2253-1bab-470b-9395-b0a8302db2db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:16:39.728008 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932a2253-1bab-470b-9395-b0a8302db2db None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:16:39.728808 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932a2253-1bab-470b-9395-b0a8302db2db None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:16:39.729200 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932a2253-1bab-470b-9395-b0a8302db2db None admin] PATH_INFO: `/v3/projects/a88c6977d0554a1eab3e30118ede2f46` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:16:39.730924 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932a2253-1bab-470b-9395-b0a8302db2db None admin] RBAC: Authorizing `identity:get_project(project_id=a88c6977d0554a1eab3e30118ede2f46)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:16:39.732906 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932a2253-1bab-470b-9395-b0a8302db2db None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:16:39.735341 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 61/124] 10.222.0.30 () {56 vars in 1186 bytes} [Tue Sep 6 04:16:39 2022] GET /identity/v3/projects/a88c6977d0554a1eab3e30118ede2f46 => generated 298 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:16:39.741749 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932a2253-1bab-470b-9395-b0a8302db2db None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:16:39.755011 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c19be41-e6b4-4fb8-b86d-b4522da5fbc2 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:16:39.795516 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d5266a-647b-49f6-b070-a2cb76acd0ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:16:39.796480 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d5266a-647b-49f6-b070-a2cb76acd0ec None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:16:39.796868 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d5266a-647b-49f6-b070-a2cb76acd0ec None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:16:39.797383 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d5266a-647b-49f6-b070-a2cb76acd0ec None admin] PATH_INFO: `/v3/users/46380000cf7c43d197c6d8207cc91e2a` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:16:39.798636 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d5266a-647b-49f6-b070-a2cb76acd0ec None admin] RBAC: Authorizing `identity:get_user(user_id=46380000cf7c43d197c6d8207cc91e2a)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:16:39.800290 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d5266a-647b-49f6-b070-a2cb76acd0ec None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:16:39.802142 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 64/125] 10.222.0.30 () {56 vars in 1177 bytes} [Tue Sep 6 04:16:39 2022] GET /identity/v3/users/46380000cf7c43d197c6d8207cc91e2a => generated 275 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:16:39.803771 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d5266a-647b-49f6-b070-a2cb76acd0ec None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:16:39.851903 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932a2253-1bab-470b-9395-b0a8302db2db None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:16:39.889153 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057b3f9f-bae2-49a3-8d7c-6efd31d0f1fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:16:39.889832 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057b3f9f-bae2-49a3-8d7c-6efd31d0f1fe None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:16:39.890075 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057b3f9f-bae2-49a3-8d7c-6efd31d0f1fe None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:16:39.890348 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057b3f9f-bae2-49a3-8d7c-6efd31d0f1fe None admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:16:39.891020 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057b3f9f-bae2-49a3-8d7c-6efd31d0f1fe None admin] RBAC: Adding query filter params (name=admin) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:16:39.891323 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057b3f9f-bae2-49a3-8d7c-6efd31d0f1fe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:16:39.892779 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057b3f9f-bae2-49a3-8d7c-6efd31d0f1fe None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:16:39.901542 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 62/126] 10.222.0.30 () {56 vars in 1099 bytes} [Tue Sep 6 04:16:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:16:39.902354 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057b3f9f-bae2-49a3-8d7c-6efd31d0f1fe None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:16:39.908350 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:16:39.918475 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d5266a-647b-49f6-b070-a2cb76acd0ec None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:16:39.924715 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:16:39.956767 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7403fea4-1519-4b31-9d30-c70ce0812316 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:16:39.957676 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7403fea4-1519-4b31-9d30-c70ce0812316 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:16:39.958099 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7403fea4-1519-4b31-9d30-c70ce0812316 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:16:39.958562 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7403fea4-1519-4b31-9d30-c70ce0812316 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:16:39.977612 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7403fea4-1519-4b31-9d30-c70ce0812316 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 04:16:39.983001 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-7403fea4-1519-4b31-9d30-c70ce0812316 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 06 04:16:39.983001 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 04:16:39.983001 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 04:16:39.983001 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 04:16:39.983001 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 04:16:39.983001 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 04:16:39.983001 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 04:16:39.983001 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 04:16:39.983001 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 04:16:39.983001 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 04:16:39.983001 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 04:16:39.983001 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 04:16:39.983001 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 04:16:39.983001 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 04:16:39.983001 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 04:16:39.983001 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 04:16:39.984081 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 04:16:39.984081 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 04:16:39.984081 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 04:16:39.984081 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 04:16:39.984081 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 04:16:39.984081 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 04:16:39.984081 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 04:16:39.984081 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 04:16:39.984081 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 04:16:39.984081 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 04:16:39.984081 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 04:16:39.984081 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 04:16:39.984081 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 04:16:39.984081 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 04:16:39.984081 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 04:16:39.984081 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 04:16:39.984981 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 04:16:39.984981 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 04:16:39.984981 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 04:16:39.984981 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 04:16:39.984981 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 04:16:39.984981 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 04:16:39.984981 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 04:16:39.984981 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 04:16:39.984981 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 04:16:39.984981 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 04:16:39.984981 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 06 04:16:39.984981 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 04:16:39.993217 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 65/127] 10.222.0.30 () {56 vars in 1099 bytes} [Tue Sep 6 04:16:39 2022] GET /identity/v3/roles/member => generated 84 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 04:16:39.994491 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7403fea4-1519-4b31-9d30-c70ce0812316 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:16:40.009570 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057b3f9f-bae2-49a3-8d7c-6efd31d0f1fe None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:16:40.044974 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edafcf0d-fcc6-4c8d-b27e-3b3abbe1e4e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:16:40.045679 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edafcf0d-fcc6-4c8d-b27e-3b3abbe1e4e6 None admin] REQUEST_METHOD: `PUT` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:16:40.045961 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edafcf0d-fcc6-4c8d-b27e-3b3abbe1e4e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:16:40.046235 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edafcf0d-fcc6-4c8d-b27e-3b3abbe1e4e6 None admin] PATH_INFO: `/v3/projects/a88c6977d0554a1eab3e30118ede2f46/users/46380000cf7c43d197c6d8207cc91e2a/roles/29607b2de39f40d68bb6ceef64b8a732` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:16:40.048168 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edafcf0d-fcc6-4c8d-b27e-3b3abbe1e4e6 None admin] RBAC: Authorizing `identity:create_grant(project_id=a88c6977d0554a1eab3e30118ede2f46, user_id=46380000cf7c43d197c6d8207cc91e2a, role_id=29607b2de39f40d68bb6ceef64b8a732)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:16:40.049659 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edafcf0d-fcc6-4c8d-b27e-3b3abbe1e4e6 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:16:40.115274 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7403fea4-1519-4b31-9d30-c70ce0812316 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:16:40.120588 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 63/128] 10.222.0.30 () {58 vars in 1439 bytes} [Tue Sep 6 04:16:39 2022] PUT /identity/v3/projects/a88c6977d0554a1eab3e30118ede2f46/users/46380000cf7c43d197c6d8207cc91e2a/roles/29607b2de39f40d68bb6ceef64b8a732 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 04:16:40.126368 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edafcf0d-fcc6-4c8d-b27e-3b3abbe1e4e6 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:16:40.188496 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7dcd40-f954-4a5b-a964-0ebf2319a7db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:16:40.189164 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7dcd40-f954-4a5b-a964-0ebf2319a7db None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:16:40.189475 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7dcd40-f954-4a5b-a964-0ebf2319a7db None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:16:40.189756 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7dcd40-f954-4a5b-a964-0ebf2319a7db None admin] PATH_INFO: `/v3/projects/a88c6977d0554a1eab3e30118ede2f46` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:16:40.192984 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7dcd40-f954-4a5b-a964-0ebf2319a7db None admin] RBAC: Authorizing `identity:get_project(project_id=a88c6977d0554a1eab3e30118ede2f46)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:16:40.197290 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7dcd40-f954-4a5b-a964-0ebf2319a7db None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:16:40.200743 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 66/129] 10.222.0.30 () {56 vars in 1186 bytes} [Tue Sep 6 04:16:39 2022] GET /identity/v3/projects/a88c6977d0554a1eab3e30118ede2f46 => generated 298 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:16:40.204045 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7dcd40-f954-4a5b-a964-0ebf2319a7db None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:16:40.251843 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:16:40.317791 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:16:40.340432 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edafcf0d-fcc6-4c8d-b27e-3b3abbe1e4e6 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:16:40.369365 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7dcd40-f954-4a5b-a964-0ebf2319a7db None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:16:40.394139 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e36135b-4d7f-4eed-bad4-1afd225cd107 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:16:40.394882 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e36135b-4d7f-4eed-bad4-1afd225cd107 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:16:40.395262 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e36135b-4d7f-4eed-bad4-1afd225cd107 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:16:40.395539 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e36135b-4d7f-4eed-bad4-1afd225cd107 None admin] PATH_INFO: `/v3/users/b7c5062264774b85bf945fb9ac7602db` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:16:40.398094 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e36135b-4d7f-4eed-bad4-1afd225cd107 None admin] RBAC: Authorizing `identity:get_user(user_id=b7c5062264774b85bf945fb9ac7602db)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:16:40.399830 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e36135b-4d7f-4eed-bad4-1afd225cd107 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:16:40.401494 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 64/130] 10.222.0.30 () {56 vars in 1177 bytes} [Tue Sep 6 04:16:40 2022] GET /identity/v3/users/b7c5062264774b85bf945fb9ac7602db => generated 247 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:16:40.402449 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e36135b-4d7f-4eed-bad4-1afd225cd107 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:16:40.407837 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6fb020e-e172-43ed-b377-e85108737f7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:16:40.408577 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fb020e-e172-43ed-b377-e85108737f7f None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:16:40.408976 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fb020e-e172-43ed-b377-e85108737f7f None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:16:40.409372 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fb020e-e172-43ed-b377-e85108737f7f None admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:16:40.410143 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6fb020e-e172-43ed-b377-e85108737f7f None admin] RBAC: Adding query filter params (name=member) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:16:40.410545 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6fb020e-e172-43ed-b377-e85108737f7f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:16:40.412407 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6fb020e-e172-43ed-b377-e85108737f7f None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:16:40.422360 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 67/131] 10.222.0.30 () {56 vars in 1101 bytes} [Tue Sep 6 04:16:40 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:16:40.423327 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6fb020e-e172-43ed-b377-e85108737f7f None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:16:40.565664 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:16:40.583074 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6fb020e-e172-43ed-b377-e85108737f7f None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:16:40.623479 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4401b7-d36f-4a85-8492-15d46d7180d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:16:40.624174 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4401b7-d36f-4a85-8492-15d46d7180d4 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:16:40.624463 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4401b7-d36f-4a85-8492-15d46d7180d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:16:40.624793 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4401b7-d36f-4a85-8492-15d46d7180d4 None admin] PATH_INFO: `/v3/projects/a88c6977d0554a1eab3e30118ede2f46` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:16:40.626475 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4401b7-d36f-4a85-8492-15d46d7180d4 None admin] RBAC: Authorizing `identity:get_project(project_id=a88c6977d0554a1eab3e30118ede2f46)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:16:40.628312 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4401b7-d36f-4a85-8492-15d46d7180d4 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:16:40.630001 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 68/132] 10.222.0.30 () {56 vars in 1186 bytes} [Tue Sep 6 04:16:40 2022] GET /identity/v3/projects/a88c6977d0554a1eab3e30118ede2f46 => generated 298 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:16:40.630774 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4401b7-d36f-4a85-8492-15d46d7180d4 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:16:40.729703 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e36135b-4d7f-4eed-bad4-1afd225cd107 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:16:40.750613 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4401b7-d36f-4a85-8492-15d46d7180d4 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:16:40.766233 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5297eb5a-524e-483c-a6df-afd6c27e3b17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:16:40.766912 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5297eb5a-524e-483c-a6df-afd6c27e3b17 None admin] REQUEST_METHOD: `PUT` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:16:40.767198 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5297eb5a-524e-483c-a6df-afd6c27e3b17 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:16:40.767478 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5297eb5a-524e-483c-a6df-afd6c27e3b17 None admin] PATH_INFO: `/v3/projects/a88c6977d0554a1eab3e30118ede2f46/users/46380000cf7c43d197c6d8207cc91e2a/roles/77197446dd2541b9a4fab9a654c08fc9` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:16:40.769776 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5297eb5a-524e-483c-a6df-afd6c27e3b17 None admin] RBAC: Authorizing `identity:create_grant(project_id=a88c6977d0554a1eab3e30118ede2f46, user_id=46380000cf7c43d197c6d8207cc91e2a, role_id=77197446dd2541b9a4fab9a654c08fc9)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:16:40.771237 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5297eb5a-524e-483c-a6df-afd6c27e3b17 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:16:40.794689 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19efbc3-1897-49c7-91e9-3587e1b2d9a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:16:40.795340 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19efbc3-1897-49c7-91e9-3587e1b2d9a7 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:16:40.795628 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19efbc3-1897-49c7-91e9-3587e1b2d9a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:16:40.795904 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19efbc3-1897-49c7-91e9-3587e1b2d9a7 None admin] PATH_INFO: `/v3/users/46380000cf7c43d197c6d8207cc91e2a` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:16:40.797088 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19efbc3-1897-49c7-91e9-3587e1b2d9a7 None admin] RBAC: Authorizing `identity:get_user(user_id=46380000cf7c43d197c6d8207cc91e2a)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:16:40.798579 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19efbc3-1897-49c7-91e9-3587e1b2d9a7 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:16:40.804925 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 69/133] 10.222.0.30 () {56 vars in 1177 bytes} [Tue Sep 6 04:16:40 2022] GET /identity/v3/users/46380000cf7c43d197c6d8207cc91e2a => generated 275 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:16:40.804925 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19efbc3-1897-49c7-91e9-3587e1b2d9a7 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:16:40.825652 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:16:40.889347 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 65/134] 10.222.0.30 () {58 vars in 1439 bytes} [Tue Sep 6 04:16:40 2022] PUT /identity/v3/projects/a88c6977d0554a1eab3e30118ede2f46/users/46380000cf7c43d197c6d8207cc91e2a/roles/77197446dd2541b9a4fab9a654c08fc9 => generated 0 bytes in 487 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 04:16:40.899964 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5baf214f-f3d9-46e1-a4b4-3d1856b93d9a None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:16:40.900570 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5baf214f-f3d9-46e1-a4b4-3d1856b93d9a None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:16:40.901048 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5baf214f-f3d9-46e1-a4b4-3d1856b93d9a None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:16:40.902621 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 66/135] 10.222.0.30 () {54 vars in 927 bytes} [Tue Sep 6 04:16:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 04:16:40.903520 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5baf214f-f3d9-46e1-a4b4-3d1856b93d9a None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:16:40.912559 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19efbc3-1897-49c7-91e9-3587e1b2d9a7 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:16:40.929392 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:16:40.964226 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5a38b9-7002-456c-b8f5-43cef43ead82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:16:40.966544 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5a38b9-7002-456c-b8f5-43cef43ead82 None admin] REQUEST_METHOD: `PUT` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:16:40.967450 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5a38b9-7002-456c-b8f5-43cef43ead82 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:16:40.967875 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5a38b9-7002-456c-b8f5-43cef43ead82 None admin] PATH_INFO: `/v3/projects/a88c6977d0554a1eab3e30118ede2f46/users/b7c5062264774b85bf945fb9ac7602db/roles/ea4c814713b149108e488de855b025e4` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:16:40.972085 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5a38b9-7002-456c-b8f5-43cef43ead82 None admin] RBAC: Authorizing `identity:create_grant(project_id=a88c6977d0554a1eab3e30118ede2f46, user_id=b7c5062264774b85bf945fb9ac7602db, role_id=ea4c814713b149108e488de855b025e4)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:16:40.974523 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5a38b9-7002-456c-b8f5-43cef43ead82 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:16:41.055646 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 70/136] 10.222.0.30 () {58 vars in 1439 bytes} [Tue Sep 6 04:16:40 2022] PUT /identity/v3/projects/a88c6977d0554a1eab3e30118ede2f46/users/b7c5062264774b85bf945fb9ac7602db/roles/ea4c814713b149108e488de855b025e4 => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 04:16:41.057748 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698c5d4b-a323-4b7a-b9a7-9866072b3607 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:16:41.058239 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698c5d4b-a323-4b7a-b9a7-9866072b3607 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:16:41.058687 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698c5d4b-a323-4b7a-b9a7-9866072b3607 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:16:41.099876 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-698c5d4b-a323-4b7a-b9a7-9866072b3607 None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:16:41.101146 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-698c5d4b-a323-4b7a-b9a7-9866072b3607 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:16:41.107859 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5baf214f-f3d9-46e1-a4b4-3d1856b93d9a None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:16:41.184470 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65fab44-2215-44f2-8f25-d180f530c2b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:16:41.186700 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65fab44-2215-44f2-8f25-d180f530c2b0 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:16:41.187731 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65fab44-2215-44f2-8f25-d180f530c2b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:16:41.188646 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65fab44-2215-44f2-8f25-d180f530c2b0 None admin] PATH_INFO: `/v3/projects/194c60f0a143489a81851796cf870320` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:16:41.192131 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65fab44-2215-44f2-8f25-d180f530c2b0 None admin] RBAC: Authorizing `identity:get_project(project_id=194c60f0a143489a81851796cf870320)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:16:41.193911 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65fab44-2215-44f2-8f25-d180f530c2b0 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:16:41.200950 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 67/137] 10.222.0.30 () {56 vars in 1186 bytes} [Tue Sep 6 04:16:40 2022] GET /identity/v3/projects/194c60f0a143489a81851796cf870320 => generated 312 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:16:41.206873 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65fab44-2215-44f2-8f25-d180f530c2b0 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:16:41.259841 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 71/138] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Sep 6 04:16:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 04:16:41.336090 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:16:41.365265 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65fab44-2215-44f2-8f25-d180f530c2b0 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:16:41.418180 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191f33ca-9e14-4636-b9ef-a568cd985aff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:16:41.418878 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191f33ca-9e14-4636-b9ef-a568cd985aff None admin] REQUEST_METHOD: `PUT` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:16:41.419166 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191f33ca-9e14-4636-b9ef-a568cd985aff None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:16:41.419448 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191f33ca-9e14-4636-b9ef-a568cd985aff None admin] PATH_INFO: `/v3/projects/194c60f0a143489a81851796cf870320/users/46380000cf7c43d197c6d8207cc91e2a/roles/29607b2de39f40d68bb6ceef64b8a732` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:16:41.421560 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-191f33ca-9e14-4636-b9ef-a568cd985aff None admin] RBAC: Authorizing `identity:create_grant(project_id=194c60f0a143489a81851796cf870320, user_id=46380000cf7c43d197c6d8207cc91e2a, role_id=29607b2de39f40d68bb6ceef64b8a732)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:16:41.423137 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-191f33ca-9e14-4636-b9ef-a568cd985aff None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:16:41.494419 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 68/139] 10.222.0.30 () {58 vars in 1439 bytes} [Tue Sep 6 04:16:41 2022] PUT /identity/v3/projects/194c60f0a143489a81851796cf870320/users/46380000cf7c43d197c6d8207cc91e2a/roles/29607b2de39f40d68bb6ceef64b8a732 => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 04:16:41.682566 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698c5d4b-a323-4b7a-b9a7-9866072b3607 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:16:41.694213 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:16:41.808924 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698c5d4b-a323-4b7a-b9a7-9866072b3607 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:16:41.848233 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480281e8-5361-4db3-966a-6929ec137a0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:16:41.848883 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480281e8-5361-4db3-966a-6929ec137a0b None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:16:41.849170 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480281e8-5361-4db3-966a-6929ec137a0b None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:16:41.849490 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480281e8-5361-4db3-966a-6929ec137a0b None admin] PATH_INFO: `/v3/domains/default` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:16:41.850678 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480281e8-5361-4db3-966a-6929ec137a0b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:16:41.852149 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480281e8-5361-4db3-966a-6929ec137a0b None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:16:41.853785 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 72/140] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Sep 6 04:16:41 2022] GET /identity/v3/domains/default => generated 199 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:16:41.859487 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191f33ca-9e14-4636-b9ef-a568cd985aff None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:16:41.909347 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5842ba7-ef96-4379-9cc7-fbf421401fc1 None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:16:41.909646 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5842ba7-ef96-4379-9cc7-fbf421401fc1 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:16:41.909906 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5842ba7-ef96-4379-9cc7-fbf421401fc1 None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:16:41.911334 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 73/141] 10.222.0.30 () {54 vars in 927 bytes} [Tue Sep 6 04:16:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 06 04:16:41.917210 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492919ea-ad5e-4fb5-be0f-bfb8640020ce None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:16:41.917516 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492919ea-ad5e-4fb5-be0f-bfb8640020ce None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:16:41.917787 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492919ea-ad5e-4fb5-be0f-bfb8640020ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:16:41.946959 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-492919ea-ad5e-4fb5-be0f-bfb8640020ce None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:16:41.948080 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-492919ea-ad5e-4fb5-be0f-bfb8640020ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:16:42.048795 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191f33ca-9e14-4636-b9ef-a568cd985aff None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:16:42.098003 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 74/142] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Sep 6 04:16:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 06 04:16:42.116852 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25498e6-4579-47bd-822a-525c518bf6eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:16:42.117501 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25498e6-4579-47bd-822a-525c518bf6eb None admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:16:42.117829 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25498e6-4579-47bd-822a-525c518bf6eb None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:16:42.118109 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25498e6-4579-47bd-822a-525c518bf6eb None admin] PATH_INFO: `/v3/users` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:16:42.118768 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25498e6-4579-47bd-822a-525c518bf6eb None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:16:42.120212 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25498e6-4579-47bd-822a-525c518bf6eb None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:16:42.137641 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:16:42.278689 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 69/143] 10.222.0.30 () {60 vars in 1132 bytes} [Tue Sep 6 04:16:41 2022] POST /identity/v3/users => generated 283 bytes in 420 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 04:16:42.662889 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492919ea-ad5e-4fb5-be0f-bfb8640020ce None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:16:42.671186 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:16:42.792867 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492919ea-ad5e-4fb5-be0f-bfb8640020ce None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:16:42.837391 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd3c78c-5987-4cb7-970a-84371251bfd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:16:42.838052 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd3c78c-5987-4cb7-970a-84371251bfd7 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:16:42.838333 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd3c78c-5987-4cb7-970a-84371251bfd7 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:16:42.838607 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd3c78c-5987-4cb7-970a-84371251bfd7 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:16:42.854174 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7195f2f6-17e5-471a-bcb7-61517b32c97e None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:16:42.854455 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7195f2f6-17e5-471a-bcb7-61517b32c97e None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:16:42.854734 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7195f2f6-17e5-471a-bcb7-61517b32c97e None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:16:42.856080 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 70/144] 10.222.0.30 () {54 vars in 927 bytes} [Tue Sep 6 04:16:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 06 04:16:42.858809 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd3c78c-5987-4cb7-970a-84371251bfd7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 04:16:42.861517 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c561ba1-67a4-4e38-ba08-8bd055593803 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:16:42.862037 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c561ba1-67a4-4e38-ba08-8bd055593803 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:16:42.862507 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c561ba1-67a4-4e38-ba08-8bd055593803 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:16:42.863699 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-acd3c78c-5987-4cb7-970a-84371251bfd7 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 06 04:16:42.863699 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 04:16:42.863699 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 04:16:42.863699 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 04:16:42.863699 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 04:16:42.863699 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 04:16:42.863699 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 04:16:42.863699 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 04:16:42.863699 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 04:16:42.863699 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 04:16:42.863699 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 04:16:42.863699 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 04:16:42.863699 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 04:16:42.863699 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 04:16:42.863699 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 04:16:42.863699 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 04:16:42.864538 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 04:16:42.864538 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 04:16:42.864538 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 04:16:42.864538 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 04:16:42.864538 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 04:16:42.864538 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 04:16:42.864538 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 04:16:42.864538 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 04:16:42.864538 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 04:16:42.864538 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 04:16:42.864538 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 04:16:42.864538 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 04:16:42.864538 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 04:16:42.864538 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 04:16:42.864538 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 04:16:42.864538 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 04:16:42.866797 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 04:16:42.866797 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 04:16:42.866797 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 04:16:42.866797 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 04:16:42.866797 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 04:16:42.866797 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 04:16:42.866797 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 04:16:42.866797 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 04:16:42.866797 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 04:16:42.866797 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 04:16:42.866797 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 06 04:16:42.866797 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 04:16:42.866797 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 75/145] 10.222.0.30 () {56 vars in 1099 bytes} [Tue Sep 6 04:16:42 2022] GET /identity/v3/roles/reader => generated 84 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 04:16:42.871202 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd3c78c-5987-4cb7-970a-84371251bfd7 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:16:42.897900 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-0c561ba1-67a4-4e38-ba08-8bd055593803 None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:16:42.900696 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-0c561ba1-67a4-4e38-ba08-8bd055593803 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:16:42.985836 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 71/146] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Sep 6 04:16:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 06 04:16:43.020271 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:16:43.076292 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd3c78c-5987-4cb7-970a-84371251bfd7 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:16:43.115233 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039bf3d4-225a-4c47-a691-e733feea1cd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:16:43.117126 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039bf3d4-225a-4c47-a691-e733feea1cd5 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:16:43.117624 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039bf3d4-225a-4c47-a691-e733feea1cd5 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:16:43.118087 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039bf3d4-225a-4c47-a691-e733feea1cd5 None admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:16:43.119021 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039bf3d4-225a-4c47-a691-e733feea1cd5 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:16:43.119513 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039bf3d4-225a-4c47-a691-e733feea1cd5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:16:43.121085 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039bf3d4-225a-4c47-a691-e733feea1cd5 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:16:43.130846 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 76/147] 10.222.0.30 () {56 vars in 1101 bytes} [Tue Sep 6 04:16:42 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 04:16:43.136254 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c561ba1-67a4-4e38-ba08-8bd055593803 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:16:43.243706 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c561ba1-67a4-4e38-ba08-8bd055593803 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:16:43.268813 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:16:43.281495 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6656ed97-27cd-4f47-b1ec-0b196cb6bef4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:16:43.282385 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6656ed97-27cd-4f47-b1ec-0b196cb6bef4 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:16:43.282833 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6656ed97-27cd-4f47-b1ec-0b196cb6bef4 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:16:43.283815 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6656ed97-27cd-4f47-b1ec-0b196cb6bef4 None admin] PATH_INFO: `/v3/users/501af79e7c0d4d768256c193a0305538` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:16:43.284932 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6656ed97-27cd-4f47-b1ec-0b196cb6bef4 None admin] RBAC: Authorizing `identity:get_user(user_id=501af79e7c0d4d768256c193a0305538)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:16:43.286613 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6656ed97-27cd-4f47-b1ec-0b196cb6bef4 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:16:43.289494 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 72/148] 10.222.0.30 () {56 vars in 1177 bytes} [Tue Sep 6 04:16:43 2022] GET /identity/v3/users/501af79e7c0d4d768256c193a0305538 => generated 289 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:16:43.294675 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039bf3d4-225a-4c47-a691-e733feea1cd5 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:16:43.408934 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039bf3d4-225a-4c47-a691-e733feea1cd5 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:16:43.438838 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:16:43.447569 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5274ef-8113-49b1-8703-346fec379ca3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:16:43.448206 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5274ef-8113-49b1-8703-346fec379ca3 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:16:43.448483 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5274ef-8113-49b1-8703-346fec379ca3 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:16:43.448754 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5274ef-8113-49b1-8703-346fec379ca3 None admin] PATH_INFO: `/v3/projects/a88c6977d0554a1eab3e30118ede2f46` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:16:43.449918 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5274ef-8113-49b1-8703-346fec379ca3 None admin] RBAC: Authorizing `identity:get_project(project_id=a88c6977d0554a1eab3e30118ede2f46)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:16:43.451616 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5274ef-8113-49b1-8703-346fec379ca3 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:16:43.453417 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 77/149] 10.222.0.30 () {56 vars in 1186 bytes} [Tue Sep 6 04:16:43 2022] GET /identity/v3/projects/a88c6977d0554a1eab3e30118ede2f46 => generated 298 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:16:43.460433 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6656ed97-27cd-4f47-b1ec-0b196cb6bef4 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:16:43.541084 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5274ef-8113-49b1-8703-346fec379ca3 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:16:43.656511 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6656ed97-27cd-4f47-b1ec-0b196cb6bef4 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:16:43.693344 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5274ef-8113-49b1-8703-346fec379ca3 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:16:43.715045 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9d09d6-00e1-4999-ad9b-688b1e844dc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:16:43.716068 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9d09d6-00e1-4999-ad9b-688b1e844dc9 None admin] REQUEST_METHOD: `PUT` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:16:43.716527 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9d09d6-00e1-4999-ad9b-688b1e844dc9 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:16:43.716978 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9d09d6-00e1-4999-ad9b-688b1e844dc9 None admin] PATH_INFO: `/v3/projects/a88c6977d0554a1eab3e30118ede2f46/users/501af79e7c0d4d768256c193a0305538/roles/3c011524d1df425b902b8b32c1b485c3` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:16:43.722614 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b9d09d6-00e1-4999-ad9b-688b1e844dc9 None admin] RBAC: Authorizing `identity:create_grant(project_id=a88c6977d0554a1eab3e30118ede2f46, user_id=501af79e7c0d4d768256c193a0305538, role_id=3c011524d1df425b902b8b32c1b485c3)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:16:43.724264 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b9d09d6-00e1-4999-ad9b-688b1e844dc9 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:16:43.729015 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce87c11-06ea-4324-b448-b25c95a99cca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:16:43.731999 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce87c11-06ea-4324-b448-b25c95a99cca None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:16:43.731999 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce87c11-06ea-4324-b448-b25c95a99cca None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:16:43.731999 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce87c11-06ea-4324-b448-b25c95a99cca None admin] PATH_INFO: `/v3/roles/member` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:16:43.742745 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce87c11-06ea-4324-b448-b25c95a99cca None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 04:16:43.748408 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-3ce87c11-06ea-4324-b448-b25c95a99cca None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 06 04:16:43.748408 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 04:16:43.748408 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 04:16:43.748408 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 04:16:43.748408 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 04:16:43.748408 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 04:16:43.748408 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 04:16:43.748408 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 04:16:43.748408 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 04:16:43.748408 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 04:16:43.748408 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 04:16:43.748408 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 04:16:43.748408 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 04:16:43.748408 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 04:16:43.748408 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 04:16:43.748408 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 04:16:43.749407 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 04:16:43.749407 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 04:16:43.749407 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 04:16:43.749407 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 04:16:43.749407 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 04:16:43.749407 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 04:16:43.749407 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 04:16:43.749407 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 04:16:43.749407 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 04:16:43.749407 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 04:16:43.749407 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 04:16:43.749407 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 04:16:43.749407 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 04:16:43.749407 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 04:16:43.749407 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 04:16:43.749407 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 04:16:43.750197 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 04:16:43.750197 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 04:16:43.750197 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 04:16:43.750197 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 04:16:43.750197 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 04:16:43.750197 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 04:16:43.750197 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 04:16:43.750197 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 04:16:43.750197 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 04:16:43.750197 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 04:16:43.750197 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 06 04:16:43.750197 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 04:16:43.750197 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 78/150] 10.222.0.30 () {56 vars in 1099 bytes} [Tue Sep 6 04:16:43 2022] GET /identity/v3/roles/member => generated 84 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 04:16:43.751599 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01fa8b8-decd-400a-ad34-90f8926a6c70 None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:16:43.752009 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01fa8b8-decd-400a-ad34-90f8926a6c70 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:16:43.752346 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01fa8b8-decd-400a-ad34-90f8926a6c70 None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:16:43.757979 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 79/151] 10.222.0.30 () {54 vars in 927 bytes} [Tue Sep 6 04:16:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 04:16:43.758969 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01fa8b8-decd-400a-ad34-90f8926a6c70 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:16:43.826202 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:16:43.835916 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 73/152] 10.222.0.30 () {58 vars in 1439 bytes} [Tue Sep 6 04:16:43 2022] PUT /identity/v3/projects/a88c6977d0554a1eab3e30118ede2f46/users/501af79e7c0d4d768256c193a0305538/roles/3c011524d1df425b902b8b32c1b485c3 => generated 0 bytes in 377 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 04:16:43.837587 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64dd958d-da7f-4e1c-a17c-f552c44c0656 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:16:43.838083 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64dd958d-da7f-4e1c-a17c-f552c44c0656 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:16:43.838580 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64dd958d-da7f-4e1c-a17c-f552c44c0656 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:16:43.879415 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01fa8b8-decd-400a-ad34-90f8926a6c70 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:16:43.886423 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-64dd958d-da7f-4e1c-a17c-f552c44c0656 None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:16:43.887664 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-64dd958d-da7f-4e1c-a17c-f552c44c0656 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:16:43.916919 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64801361-12d9-4b97-9286-f901c3a35455 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:16:43.918777 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64801361-12d9-4b97-9286-f901c3a35455 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:16:43.919211 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64801361-12d9-4b97-9286-f901c3a35455 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:16:43.919622 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64801361-12d9-4b97-9286-f901c3a35455 None admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:16:43.920456 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64801361-12d9-4b97-9286-f901c3a35455 None admin] RBAC: Adding query filter params (name=member) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:16:43.920900 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64801361-12d9-4b97-9286-f901c3a35455 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:16:43.922534 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64801361-12d9-4b97-9286-f901c3a35455 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:16:43.942500 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 80/153] 10.222.0.30 () {56 vars in 1101 bytes} [Tue Sep 6 04:16:43 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:16:43.944217 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2540982-5181-48b6-bdb1-347622a88dbb None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:16:43.944744 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2540982-5181-48b6-bdb1-347622a88dbb None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:16:43.945155 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2540982-5181-48b6-bdb1-347622a88dbb None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:16:43.946885 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 81/154] 10.222.0.30 () {54 vars in 927 bytes} [Tue Sep 6 04:16:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 04:16:43.954697 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a3f022-ab0b-41f7-ba1f-43338610e65b None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:16:43.955269 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a3f022-ab0b-41f7-ba1f-43338610e65b None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:16:43.955699 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a3f022-ab0b-41f7-ba1f-43338610e65b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:16:43.971706 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:16:43.981542 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-a2a3f022-ab0b-41f7-ba1f-43338610e65b None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:16:43.983029 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-a2a3f022-ab0b-41f7-ba1f-43338610e65b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:16:44.047668 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 74/155] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Sep 6 04:16:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 04:16:44.048556 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64dd958d-da7f-4e1c-a17c-f552c44c0656 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:16:44.079754 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 82/156] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Sep 6 04:16:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 06 04:16:44.083219 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581714dc-0a29-4f78-87a1-44324fb87d40 None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:16:44.083692 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581714dc-0a29-4f78-87a1-44324fb87d40 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:16:44.084675 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581714dc-0a29-4f78-87a1-44324fb87d40 None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:16:44.086859 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 83/157] 10.222.0.30 () {54 vars in 927 bytes} [Tue Sep 6 04:16:44 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 04:16:44.092960 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fba6602-5c1b-4712-bf58-6d12170575b7 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:16:44.093495 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fba6602-5c1b-4712-bf58-6d12170575b7 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:16:44.093923 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fba6602-5c1b-4712-bf58-6d12170575b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:16:44.126422 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-0fba6602-5c1b-4712-bf58-6d12170575b7 None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:16:44.128677 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-0fba6602-5c1b-4712-bf58-6d12170575b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:16:44.228984 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:16:44.284467 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64dd958d-da7f-4e1c-a17c-f552c44c0656 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:16:44.298637 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 84/158] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Sep 6 04:16:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 04:16:44.346652 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66330f2d-0a66-4934-996b-8a97fe5b1ed9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:16:44.347231 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66330f2d-0a66-4934-996b-8a97fe5b1ed9 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:16:44.347516 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66330f2d-0a66-4934-996b-8a97fe5b1ed9 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:16:44.347797 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66330f2d-0a66-4934-996b-8a97fe5b1ed9 None admin] PATH_INFO: `/v3/users/46380000cf7c43d197c6d8207cc91e2a` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:16:44.348968 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66330f2d-0a66-4934-996b-8a97fe5b1ed9 None admin] RBAC: Authorizing `identity:get_user(user_id=46380000cf7c43d197c6d8207cc91e2a)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:16:44.351412 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66330f2d-0a66-4934-996b-8a97fe5b1ed9 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:16:44.354007 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 75/159] 10.222.0.30 () {56 vars in 1177 bytes} [Tue Sep 6 04:16:44 2022] GET /identity/v3/users/46380000cf7c43d197c6d8207cc91e2a => generated 275 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:16:44.358970 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fba6602-5c1b-4712-bf58-6d12170575b7 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:16:44.473544 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66330f2d-0a66-4934-996b-8a97fe5b1ed9 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:16:44.513069 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fba6602-5c1b-4712-bf58-6d12170575b7 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:16:44.545591 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:16:44.610116 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e100d83d-59b9-4e8f-8ad5-c758049efc3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:16:44.610671 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e100d83d-59b9-4e8f-8ad5-c758049efc3e None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:16:44.610954 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e100d83d-59b9-4e8f-8ad5-c758049efc3e None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:16:44.611226 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e100d83d-59b9-4e8f-8ad5-c758049efc3e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:16:44.649841 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66330f2d-0a66-4934-996b-8a97fe5b1ed9 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:16:44.702884 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:16:44.730609 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f82d1f0-f996-4f5d-bf97-e723874ae737 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:16:44.731273 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f82d1f0-f996-4f5d-bf97-e723874ae737 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:16:44.731562 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f82d1f0-f996-4f5d-bf97-e723874ae737 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:16:44.731842 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f82d1f0-f996-4f5d-bf97-e723874ae737 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:16:44.732256 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e100d83d-59b9-4e8f-8ad5-c758049efc3e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:16:44.733924 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e100d83d-59b9-4e8f-8ad5-c758049efc3e None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:16:44.761985 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f82d1f0-f996-4f5d-bf97-e723874ae737 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 04:16:44.767047 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-2f82d1f0-f996-4f5d-bf97-e723874ae737 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 06 04:16:44.767047 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 04:16:44.767047 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 04:16:44.767047 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 04:16:44.767047 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 04:16:44.767047 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 04:16:44.767047 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 04:16:44.767047 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 04:16:44.767047 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 04:16:44.767047 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 04:16:44.767047 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 04:16:44.767047 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 04:16:44.767047 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 04:16:44.767047 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 04:16:44.767047 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 04:16:44.767047 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 04:16:44.767917 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 04:16:44.767917 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 04:16:44.767917 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 04:16:44.767917 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 04:16:44.767917 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 04:16:44.767917 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 04:16:44.767917 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 04:16:44.767917 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 04:16:44.767917 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 04:16:44.767917 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 04:16:44.767917 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 04:16:44.767917 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 04:16:44.767917 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 04:16:44.767917 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 04:16:44.767917 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 04:16:44.767917 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 04:16:44.768734 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 04:16:44.768734 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 04:16:44.768734 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 04:16:44.768734 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 04:16:44.768734 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 04:16:44.768734 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 04:16:44.768734 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 04:16:44.768734 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 04:16:44.768734 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 04:16:44.768734 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 04:16:44.768734 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 06 04:16:44.768734 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 04:16:44.768734 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 76/160] 10.222.0.30 () {56 vars in 1114 bytes} [Tue Sep 6 04:16:44 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 295 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 04:16:44.769400 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f82d1f0-f996-4f5d-bf97-e723874ae737 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:16:44.884899 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 85/161] 10.222.0.30 () {58 vars in 1282 bytes} [Tue Sep 6 04:16:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 526 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 06 04:16:44.885523 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e100d83d-59b9-4e8f-8ad5-c758049efc3e None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:16:44.972592 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f82d1f0-f996-4f5d-bf97-e723874ae737 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:16:44.985499 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:16:45.023846 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:16:45.040880 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2ba3f0-1dde-4921-bf87-fe86b6d88a2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:16:45.043259 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2ba3f0-1dde-4921-bf87-fe86b6d88a2b None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:16:45.043259 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2ba3f0-1dde-4921-bf87-fe86b6d88a2b None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:16:45.043259 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2ba3f0-1dde-4921-bf87-fe86b6d88a2b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:16:45.064463 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce2ba3f0-1dde-4921-bf87-fe86b6d88a2b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 04:16:45.068866 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e100d83d-59b9-4e8f-8ad5-c758049efc3e None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:16:45.072543 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-ce2ba3f0-1dde-4921-bf87-fe86b6d88a2b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 04:16:45.072543 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 04:16:45.072543 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 04:16:45.072543 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 04:16:45.072543 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 04:16:45.072543 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 04:16:45.072543 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 04:16:45.072543 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 04:16:45.072543 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 04:16:45.072543 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 04:16:45.072543 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 04:16:45.072543 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 04:16:45.072543 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 04:16:45.072543 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 04:16:45.072543 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 04:16:45.072543 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 04:16:45.073394 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 04:16:45.073394 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 04:16:45.073394 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 04:16:45.073394 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 04:16:45.073394 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 04:16:45.073394 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 04:16:45.073394 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 04:16:45.073394 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 04:16:45.073394 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 04:16:45.073394 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 04:16:45.073394 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 04:16:45.073394 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 04:16:45.073394 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 04:16:45.073394 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 04:16:45.073394 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 04:16:45.073394 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 04:16:45.074164 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 04:16:45.074164 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 04:16:45.074164 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 04:16:45.074164 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 04:16:45.074164 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 04:16:45.074164 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 04:16:45.074164 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 04:16:45.074164 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 04:16:45.074164 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 04:16:45.074164 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 04:16:45.074164 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 04:16:45.074164 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 04:16:45.074164 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 77/162] 10.222.0.30 () {56 vars in 1096 bytes} [Tue Sep 6 04:16:44 2022] GET /identity/v3/roles/admin => generated 83 bytes in 305 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 04:16:45.082111 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2ba3f0-1dde-4921-bf87-fe86b6d88a2b None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:16:45.136970 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651d8c28-fe29-4581-9a05-30d1fd66b5e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:16:45.138341 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651d8c28-fe29-4581-9a05-30d1fd66b5e9 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:16:45.139854 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651d8c28-fe29-4581-9a05-30d1fd66b5e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:16:45.141390 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651d8c28-fe29-4581-9a05-30d1fd66b5e9 None admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:16:45.142272 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651d8c28-fe29-4581-9a05-30d1fd66b5e9 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:16:45.144751 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651d8c28-fe29-4581-9a05-30d1fd66b5e9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:16:45.147234 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651d8c28-fe29-4581-9a05-30d1fd66b5e9 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:16:45.167527 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 86/163] 10.222.0.30 () {56 vars in 1111 bytes} [Tue Sep 6 04:16:44 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:16:45.168565 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651d8c28-fe29-4581-9a05-30d1fd66b5e9 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:16:45.224434 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2ba3f0-1dde-4921-bf87-fe86b6d88a2b None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:16:45.260265 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef54f6c-5946-414b-bab6-cac77b98f84e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:16:45.261250 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef54f6c-5946-414b-bab6-cac77b98f84e None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:16:45.261760 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef54f6c-5946-414b-bab6-cac77b98f84e None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:16:45.262222 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef54f6c-5946-414b-bab6-cac77b98f84e None admin] PATH_INFO: `/v3/roles/member` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:16:45.273113 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef54f6c-5946-414b-bab6-cac77b98f84e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 04:16:45.278604 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-5ef54f6c-5946-414b-bab6-cac77b98f84e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 06 04:16:45.278604 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 04:16:45.278604 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 04:16:45.278604 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 04:16:45.278604 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 04:16:45.278604 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 04:16:45.278604 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 04:16:45.278604 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 04:16:45.278604 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 04:16:45.278604 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 04:16:45.278604 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 04:16:45.278604 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 04:16:45.278604 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 04:16:45.278604 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 04:16:45.278604 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 04:16:45.278604 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 04:16:45.279692 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 04:16:45.279692 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 04:16:45.279692 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 04:16:45.279692 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 04:16:45.279692 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 04:16:45.279692 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 04:16:45.279692 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 04:16:45.279692 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 04:16:45.279692 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 04:16:45.279692 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 04:16:45.279692 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 04:16:45.279692 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 04:16:45.279692 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 04:16:45.279692 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 04:16:45.279692 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 04:16:45.279692 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 04:16:45.280475 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 04:16:45.280475 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 04:16:45.280475 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 04:16:45.280475 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 04:16:45.280475 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 04:16:45.280475 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 04:16:45.280475 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 04:16:45.280475 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 04:16:45.280475 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 04:16:45.280475 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 04:16:45.280475 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 06 04:16:45.280475 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 04:16:45.282762 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 78/164] 10.222.0.30 () {56 vars in 1099 bytes} [Tue Sep 6 04:16:45 2022] GET /identity/v3/roles/member => generated 84 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 04:16:45.283749 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef54f6c-5946-414b-bab6-cac77b98f84e None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:16:45.314737 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:16:45.391852 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef54f6c-5946-414b-bab6-cac77b98f84e None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:16:45.399595 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651d8c28-fe29-4581-9a05-30d1fd66b5e9 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:16:45.434692 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9939388-3f1b-4a73-b130-19732a3dd583 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:16:45.435313 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9939388-3f1b-4a73-b130-19732a3dd583 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:16:45.435595 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9939388-3f1b-4a73-b130-19732a3dd583 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:16:45.435865 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9939388-3f1b-4a73-b130-19732a3dd583 None admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:16:45.436532 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9939388-3f1b-4a73-b130-19732a3dd583 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:16:45.436833 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9939388-3f1b-4a73-b130-19732a3dd583 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:16:45.438260 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9939388-3f1b-4a73-b130-19732a3dd583 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:16:45.448698 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 79/165] 10.222.0.30 () {56 vars in 1099 bytes} [Tue Sep 6 04:16:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:16:45.449511 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9939388-3f1b-4a73-b130-19732a3dd583 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:16:45.460355 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14bb10b6-2fb3-42ea-b840-a9c0fec2d024 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:16:45.461013 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bb10b6-2fb3-42ea-b840-a9c0fec2d024 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:16:45.461291 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bb10b6-2fb3-42ea-b840-a9c0fec2d024 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:16:45.461587 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bb10b6-2fb3-42ea-b840-a9c0fec2d024 None admin] PATH_INFO: `/v3/projects/a88c6977d0554a1eab3e30118ede2f46` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:16:45.462839 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14bb10b6-2fb3-42ea-b840-a9c0fec2d024 None admin] RBAC: Authorizing `identity:get_project(project_id=a88c6977d0554a1eab3e30118ede2f46)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:16:45.464335 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14bb10b6-2fb3-42ea-b840-a9c0fec2d024 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:16:45.466137 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 87/166] 10.222.0.30 () {56 vars in 1186 bytes} [Tue Sep 6 04:16:45 2022] GET /identity/v3/projects/a88c6977d0554a1eab3e30118ede2f46 => generated 298 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:16:45.467873 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14bb10b6-2fb3-42ea-b840-a9c0fec2d024 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:16:45.486203 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:16:45.551444 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:16:45.563325 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9939388-3f1b-4a73-b130-19732a3dd583 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:16:45.603623 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bef879b-8290-4c4a-934c-cc46907d6af0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:16:45.604681 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bef879b-8290-4c4a-934c-cc46907d6af0 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:16:45.605241 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bef879b-8290-4c4a-934c-cc46907d6af0 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:16:45.605734 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bef879b-8290-4c4a-934c-cc46907d6af0 None admin] PATH_INFO: `/v3/users/46380000cf7c43d197c6d8207cc91e2a` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:16:45.607068 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bef879b-8290-4c4a-934c-cc46907d6af0 None admin] RBAC: Authorizing `identity:get_user(user_id=46380000cf7c43d197c6d8207cc91e2a)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:16:45.608823 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bef879b-8290-4c4a-934c-cc46907d6af0 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:16:45.610851 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 80/167] 10.222.0.30 () {56 vars in 1177 bytes} [Tue Sep 6 04:16:45 2022] GET /identity/v3/users/46380000cf7c43d197c6d8207cc91e2a => generated 275 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:16:45.611819 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bef879b-8290-4c4a-934c-cc46907d6af0 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:16:45.617545 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14bb10b6-2fb3-42ea-b840-a9c0fec2d024 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:16:45.676913 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af282ecf-4a0f-4428-a912-f1b134c8da10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:16:45.677510 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af282ecf-4a0f-4428-a912-f1b134c8da10 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:16:45.679053 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af282ecf-4a0f-4428-a912-f1b134c8da10 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:16:45.679053 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af282ecf-4a0f-4428-a912-f1b134c8da10 None admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:16:45.679716 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af282ecf-4a0f-4428-a912-f1b134c8da10 None admin] RBAC: Adding query filter params (name=member) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:16:45.680082 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af282ecf-4a0f-4428-a912-f1b134c8da10 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:16:45.685627 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af282ecf-4a0f-4428-a912-f1b134c8da10 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:16:45.700756 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 88/168] 10.222.0.30 () {56 vars in 1101 bytes} [Tue Sep 6 04:16:45 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:16:45.701543 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af282ecf-4a0f-4428-a912-f1b134c8da10 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:16:45.797066 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:16:45.830823 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af282ecf-4a0f-4428-a912-f1b134c8da10 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:16:45.839798 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bef879b-8290-4c4a-934c-cc46907d6af0 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:16:45.868870 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0b1a97-5261-44db-ab13-d463ee08b003 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:16:45.869534 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0b1a97-5261-44db-ab13-d463ee08b003 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:16:45.869818 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0b1a97-5261-44db-ab13-d463ee08b003 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:16:45.870093 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0b1a97-5261-44db-ab13-d463ee08b003 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:16:45.870835 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0b1a97-5261-44db-ab13-d463ee08b003 None admin] RBAC: Adding query filter params (role.id=29607b2de39f40d68bb6ceef64b8a732, scope.project.id=a88c6977d0554a1eab3e30118ede2f46, user.id=46380000cf7c43d197c6d8207cc91e2a) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:16:45.871144 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0b1a97-5261-44db-ab13-d463ee08b003 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:16:45.872615 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0b1a97-5261-44db-ab13-d463ee08b003 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:16:45.880406 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 89/169] 10.222.0.30 () {56 vars in 1374 bytes} [Tue Sep 6 04:16:45 2022] GET /identity/v3/role_assignments?user.id=46380000cf7c43d197c6d8207cc91e2a&scope.project.id=a88c6977d0554a1eab3e30118ede2f46&role.id=29607b2de39f40d68bb6ceef64b8a732 => generated 608 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:16:45.882025 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0b1a97-5261-44db-ab13-d463ee08b003 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:16:45.919039 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83016798-fc97-4aa0-a725-0dc3016b47c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:16:45.919703 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83016798-fc97-4aa0-a725-0dc3016b47c0 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:16:45.919985 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83016798-fc97-4aa0-a725-0dc3016b47c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:16:45.920260 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83016798-fc97-4aa0-a725-0dc3016b47c0 None admin] PATH_INFO: `/v3/users/b7c5062264774b85bf945fb9ac7602db` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:16:45.926002 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83016798-fc97-4aa0-a725-0dc3016b47c0 None admin] RBAC: Authorizing `identity:get_user(user_id=b7c5062264774b85bf945fb9ac7602db)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:16:45.927541 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83016798-fc97-4aa0-a725-0dc3016b47c0 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:16:45.930191 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 81/170] 10.222.0.30 () {56 vars in 1177 bytes} [Tue Sep 6 04:16:45 2022] GET /identity/v3/users/b7c5062264774b85bf945fb9ac7602db => generated 247 bytes in 318 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:16:45.930998 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83016798-fc97-4aa0-a725-0dc3016b47c0 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:16:45.980463 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:16:45.995943 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0b1a97-5261-44db-ab13-d463ee08b003 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:16:46.034314 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d745ea5-8e10-4170-8e79-ea0bd87aff71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:16:46.035129 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d745ea5-8e10-4170-8e79-ea0bd87aff71 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:16:46.035478 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d745ea5-8e10-4170-8e79-ea0bd87aff71 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:16:46.035818 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d745ea5-8e10-4170-8e79-ea0bd87aff71 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:16:46.039808 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:16:46.114654 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83016798-fc97-4aa0-a725-0dc3016b47c0 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:16:46.115129 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d745ea5-8e10-4170-8e79-ea0bd87aff71 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:16:46.116913 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d745ea5-8e10-4170-8e79-ea0bd87aff71 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:16:46.186206 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2a430f-2bb8-4b86-838e-bda9251f90ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:16:46.186794 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2a430f-2bb8-4b86-838e-bda9251f90ef None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:16:46.187076 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2a430f-2bb8-4b86-838e-bda9251f90ef None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:16:46.187351 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2a430f-2bb8-4b86-838e-bda9251f90ef None admin] PATH_INFO: `/v3/users/46380000cf7c43d197c6d8207cc91e2a` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:16:46.188606 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2a430f-2bb8-4b86-838e-bda9251f90ef None admin] RBAC: Authorizing `identity:get_user(user_id=46380000cf7c43d197c6d8207cc91e2a)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:16:46.190225 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2a430f-2bb8-4b86-838e-bda9251f90ef None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:16:46.191856 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 82/171] 10.222.0.30 () {56 vars in 1177 bytes} [Tue Sep 6 04:16:45 2022] GET /identity/v3/users/46380000cf7c43d197c6d8207cc91e2a => generated 275 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:16:46.192633 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2a430f-2bb8-4b86-838e-bda9251f90ef None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:16:46.234681 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 90/172] 10.222.0.30 () {58 vars in 1282 bytes} [Tue Sep 6 04:16:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 06 04:16:46.235662 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d745ea5-8e10-4170-8e79-ea0bd87aff71 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:16:46.277802 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:16:46.344386 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:16:46.356187 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2a430f-2bb8-4b86-838e-bda9251f90ef None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:16:46.377133 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d745ea5-8e10-4170-8e79-ea0bd87aff71 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:16:46.417363 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f967639-79a9-456d-858b-d53fd37568dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:16:46.417984 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f967639-79a9-456d-858b-d53fd37568dc None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:16:46.418259 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f967639-79a9-456d-858b-d53fd37568dc None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:16:46.418523 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f967639-79a9-456d-858b-d53fd37568dc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:16:46.426008 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26ef0cf-9e21-4d2e-a14e-2e7a159e70eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:16:46.426651 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26ef0cf-9e21-4d2e-a14e-2e7a159e70eb None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:16:46.426939 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26ef0cf-9e21-4d2e-a14e-2e7a159e70eb None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:16:46.427220 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26ef0cf-9e21-4d2e-a14e-2e7a159e70eb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:16:46.508388 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f967639-79a9-456d-858b-d53fd37568dc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:16:46.509843 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f967639-79a9-456d-858b-d53fd37568dc None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:16:46.546045 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:16:46.554554 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26ef0cf-9e21-4d2e-a14e-2e7a159e70eb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:16:46.556037 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26ef0cf-9e21-4d2e-a14e-2e7a159e70eb None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:16:46.619790 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 91/173] 10.222.0.30 () {58 vars in 1282 bytes} [Tue Sep 6 04:16:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 06 04:16:46.623405 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f967639-79a9-456d-858b-d53fd37568dc None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:16:46.731225 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f967639-79a9-456d-858b-d53fd37568dc None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:16:46.767501 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e692fa95-2df9-4ffe-9011-d80c3f7abf5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:16:46.768216 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e692fa95-2df9-4ffe-9011-d80c3f7abf5c None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:16:46.768609 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e692fa95-2df9-4ffe-9011-d80c3f7abf5c None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:16:46.768978 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e692fa95-2df9-4ffe-9011-d80c3f7abf5c None admin] PATH_INFO: `/v3/projects/a88c6977d0554a1eab3e30118ede2f46` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:16:46.770261 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e692fa95-2df9-4ffe-9011-d80c3f7abf5c None admin] RBAC: Authorizing `identity:get_project(project_id=a88c6977d0554a1eab3e30118ede2f46)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:16:46.772118 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e692fa95-2df9-4ffe-9011-d80c3f7abf5c None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:16:46.773898 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 92/174] 10.222.0.30 () {56 vars in 1186 bytes} [Tue Sep 6 04:16:46 2022] GET /identity/v3/projects/a88c6977d0554a1eab3e30118ede2f46 => generated 298 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:16:46.779008 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e692fa95-2df9-4ffe-9011-d80c3f7abf5c None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:16:46.789640 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:16:46.851823 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:16:46.876143 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 83/175] 10.222.0.30 () {58 vars in 1282 bytes} [Tue Sep 6 04:16:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 684 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 06 04:16:46.877715 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f927e89-4c4d-44ac-aa1d-3a099e5d11de None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:16:46.878009 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f927e89-4c4d-44ac-aa1d-3a099e5d11de None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:16:46.878286 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f927e89-4c4d-44ac-aa1d-3a099e5d11de None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:16:46.879540 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 84/176] 10.222.0.30 () {54 vars in 927 bytes} [Tue Sep 6 04:16:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 04:16:46.880248 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f927e89-4c4d-44ac-aa1d-3a099e5d11de None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:16:46.897151 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e692fa95-2df9-4ffe-9011-d80c3f7abf5c None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:16:46.932958 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb32ec8-98c8-422a-8eb2-162243e86024 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:16:46.933641 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb32ec8-98c8-422a-8eb2-162243e86024 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:16:46.933924 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb32ec8-98c8-422a-8eb2-162243e86024 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:16:46.934253 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb32ec8-98c8-422a-8eb2-162243e86024 None admin] PATH_INFO: `/v3/projects/194c60f0a143489a81851796cf870320` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:16:46.935411 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb32ec8-98c8-422a-8eb2-162243e86024 None admin] RBAC: Authorizing `identity:get_project(project_id=194c60f0a143489a81851796cf870320)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:16:46.936866 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb32ec8-98c8-422a-8eb2-162243e86024 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:16:46.938547 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 93/177] 10.222.0.30 () {56 vars in 1186 bytes} [Tue Sep 6 04:16:46 2022] GET /identity/v3/projects/194c60f0a143489a81851796cf870320 => generated 312 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:16:46.943909 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb32ec8-98c8-422a-8eb2-162243e86024 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:16:47.017766 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:16:47.050050 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb32ec8-98c8-422a-8eb2-162243e86024 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:16:47.087571 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2c7b2f-99e3-42c1-b1ac-4760c0b6d8c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:16:47.088315 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2c7b2f-99e3-42c1-b1ac-4760c0b6d8c3 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:16:47.088600 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2c7b2f-99e3-42c1-b1ac-4760c0b6d8c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:16:47.088874 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2c7b2f-99e3-42c1-b1ac-4760c0b6d8c3 None admin] PATH_INFO: `/v3/projects/a88c6977d0554a1eab3e30118ede2f46` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:16:47.090136 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe2c7b2f-99e3-42c1-b1ac-4760c0b6d8c3 None admin] RBAC: Authorizing `identity:get_project(project_id=a88c6977d0554a1eab3e30118ede2f46)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:16:47.091706 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe2c7b2f-99e3-42c1-b1ac-4760c0b6d8c3 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:16:47.093939 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 94/178] 10.222.0.30 () {56 vars in 1186 bytes} [Tue Sep 6 04:16:46 2022] GET /identity/v3/projects/a88c6977d0554a1eab3e30118ede2f46 => generated 298 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:16:47.095349 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6753f2d0-a407-464c-8524-901e6595816c None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:16:47.095689 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6753f2d0-a407-464c-8524-901e6595816c None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:16:47.095961 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6753f2d0-a407-464c-8524-901e6595816c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:16:47.116506 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-6753f2d0-a407-464c-8524-901e6595816c None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:16:47.117725 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-6753f2d0-a407-464c-8524-901e6595816c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:16:47.193732 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 95/179] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Sep 6 04:16:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 04:16:47.194805 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6753f2d0-a407-464c-8524-901e6595816c None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:16:47.231457 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f927e89-4c4d-44ac-aa1d-3a099e5d11de None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:16:47.266922 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84935656-d9fa-49eb-b789-0f7a6c04bcb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:16:47.267539 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84935656-d9fa-49eb-b789-0f7a6c04bcb9 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:16:47.267676 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:16:47.268159 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84935656-d9fa-49eb-b789-0f7a6c04bcb9 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:16:47.268159 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84935656-d9fa-49eb-b789-0f7a6c04bcb9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:16:47.268804 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84935656-d9fa-49eb-b789-0f7a6c04bcb9 None admin] RBAC: Adding query filter params (role.id=77197446dd2541b9a4fab9a654c08fc9, scope.project.id=a88c6977d0554a1eab3e30118ede2f46, user.id=46380000cf7c43d197c6d8207cc91e2a) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:16:47.269108 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84935656-d9fa-49eb-b789-0f7a6c04bcb9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:16:47.270594 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84935656-d9fa-49eb-b789-0f7a6c04bcb9 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:16:47.277242 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 85/180] 10.222.0.30 () {56 vars in 1374 bytes} [Tue Sep 6 04:16:46 2022] GET /identity/v3/role_assignments?user.id=46380000cf7c43d197c6d8207cc91e2a&scope.project.id=a88c6977d0554a1eab3e30118ede2f46&role.id=77197446dd2541b9a4fab9a654c08fc9 => generated 608 bytes in 397 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:16:47.279143 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84935656-d9fa-49eb-b789-0f7a6c04bcb9 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:16:47.329672 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6753f2d0-a407-464c-8524-901e6595816c None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:16:47.347918 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:16:47.381817 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8202eeaf-c26c-4799-a493-ec06a6c5b5c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:16:47.382450 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8202eeaf-c26c-4799-a493-ec06a6c5b5c1 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:16:47.382730 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8202eeaf-c26c-4799-a493-ec06a6c5b5c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:16:47.383000 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8202eeaf-c26c-4799-a493-ec06a6c5b5c1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:16:47.383756 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8202eeaf-c26c-4799-a493-ec06a6c5b5c1 None admin] RBAC: Adding query filter params (role.id=29607b2de39f40d68bb6ceef64b8a732, scope.project.id=194c60f0a143489a81851796cf870320, user.id=46380000cf7c43d197c6d8207cc91e2a) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:16:47.384065 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8202eeaf-c26c-4799-a493-ec06a6c5b5c1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:16:47.387043 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8202eeaf-c26c-4799-a493-ec06a6c5b5c1 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:16:47.398919 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 96/181] 10.222.0.30 () {56 vars in 1374 bytes} [Tue Sep 6 04:16:47 2022] GET /identity/v3/role_assignments?user.id=46380000cf7c43d197c6d8207cc91e2a&scope.project.id=194c60f0a143489a81851796cf870320&role.id=29607b2de39f40d68bb6ceef64b8a732 => generated 608 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:16:47.400644 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0224b6ac-d394-4968-9460-cca6e4c5496a None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:16:47.401291 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0224b6ac-d394-4968-9460-cca6e4c5496a None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:16:47.401963 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0224b6ac-d394-4968-9460-cca6e4c5496a None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:16:47.408273 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 97/182] 10.222.0.30 () {54 vars in 927 bytes} [Tue Sep 6 04:16:47 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 04:16:47.410606 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ce9a8a-9db1-4c09-938a-7d3119a095be None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:16:47.411371 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ce9a8a-9db1-4c09-938a-7d3119a095be None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:16:47.411836 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ce9a8a-9db1-4c09-938a-7d3119a095be None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:16:47.415173 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 98/183] 10.222.0.30 () {54 vars in 927 bytes} [Tue Sep 6 04:16:47 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 04:16:47.416807 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9cbdba-604c-4521-92f5-b23698cd5c9d None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:16:47.417257 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9cbdba-604c-4521-92f5-b23698cd5c9d None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:16:47.417746 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9cbdba-604c-4521-92f5-b23698cd5c9d None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:16:47.419505 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 99/184] 10.222.0.30 () {54 vars in 927 bytes} [Tue Sep 6 04:16:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 04:16:47.424777 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4ee1ff-1e4e-4009-9a0f-29f098acbb2e None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:16:47.424952 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84935656-d9fa-49eb-b789-0f7a6c04bcb9 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:16:47.426002 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4ee1ff-1e4e-4009-9a0f-29f098acbb2e None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:16:47.428083 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4ee1ff-1e4e-4009-9a0f-29f098acbb2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:16:47.453933 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-fb4ee1ff-1e4e-4009-9a0f-29f098acbb2e None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:16:47.455757 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-fb4ee1ff-1e4e-4009-9a0f-29f098acbb2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:16:47.464769 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed207a8-3e95-4fe9-b7ff-c023782796d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:16:47.465545 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed207a8-3e95-4fe9-b7ff-c023782796d5 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:16:47.465896 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed207a8-3e95-4fe9-b7ff-c023782796d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:16:47.466236 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed207a8-3e95-4fe9-b7ff-c023782796d5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:16:47.467098 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed207a8-3e95-4fe9-b7ff-c023782796d5 None admin] RBAC: Adding query filter params (role.id=ea4c814713b149108e488de855b025e4, scope.project.id=a88c6977d0554a1eab3e30118ede2f46, user.id=b7c5062264774b85bf945fb9ac7602db) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:16:47.467541 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed207a8-3e95-4fe9-b7ff-c023782796d5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:16:47.469152 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed207a8-3e95-4fe9-b7ff-c023782796d5 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:16:47.476809 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 86/185] 10.222.0.30 () {56 vars in 1374 bytes} [Tue Sep 6 04:16:47 2022] GET /identity/v3/role_assignments?user.id=b7c5062264774b85bf945fb9ac7602db&scope.project.id=a88c6977d0554a1eab3e30118ede2f46&role.id=ea4c814713b149108e488de855b025e4 => generated 608 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:16:47.481339 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0cfa7f-0b41-403e-aba6-93f67f093326 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:16:47.481339 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0cfa7f-0b41-403e-aba6-93f67f093326 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:16:47.481339 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0cfa7f-0b41-403e-aba6-93f67f093326 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:16:47.515801 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-3e0cfa7f-0b41-403e-aba6-93f67f093326 None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:16:47.517139 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-3e0cfa7f-0b41-403e-aba6-93f67f093326 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:16:47.574879 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 100/186] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Sep 6 04:16:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 04:16:47.581346 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee2e4e1-9079-4454-8e57-8e97ec0f8fa4 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:16:47.582888 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee2e4e1-9079-4454-8e57-8e97ec0f8fa4 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:16:47.583413 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee2e4e1-9079-4454-8e57-8e97ec0f8fa4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:16:47.620724 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-2ee2e4e1-9079-4454-8e57-8e97ec0f8fa4 None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:16:47.622200 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-2ee2e4e1-9079-4454-8e57-8e97ec0f8fa4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:16:47.636992 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 87/187] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Sep 6 04:16:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 04:16:47.637923 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0cfa7f-0b41-403e-aba6-93f67f093326 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:16:47.701857 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:16:47.706909 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:16:47.723329 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 101/188] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Sep 6 04:16:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 04:16:47.763029 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0cfa7f-0b41-403e-aba6-93f67f093326 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:16:47.803639 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80410e96-697d-4cdd-bd2b-445a355907d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:16:47.804282 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80410e96-697d-4cdd-bd2b-445a355907d1 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:16:47.804558 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80410e96-697d-4cdd-bd2b-445a355907d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:16:47.804825 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80410e96-697d-4cdd-bd2b-445a355907d1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:16:47.815795 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80410e96-697d-4cdd-bd2b-445a355907d1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 04:16:47.821462 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-80410e96-697d-4cdd-bd2b-445a355907d1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 04:16:47.821462 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 04:16:47.821462 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 04:16:47.821462 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 04:16:47.821462 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 04:16:47.821462 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 04:16:47.821462 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 04:16:47.821462 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 04:16:47.821462 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 04:16:47.821462 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 04:16:47.821462 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 04:16:47.821462 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 04:16:47.821462 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 04:16:47.821462 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 04:16:47.821462 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 04:16:47.821462 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 04:16:47.822341 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 04:16:47.822341 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 04:16:47.822341 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 04:16:47.822341 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 04:16:47.822341 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 04:16:47.822341 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 04:16:47.822341 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 04:16:47.822341 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 04:16:47.822341 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 04:16:47.822341 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 04:16:47.822341 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 04:16:47.822341 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 04:16:47.822341 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 04:16:47.822341 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 04:16:47.822341 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 04:16:47.822341 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 04:16:47.823106 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 04:16:47.823106 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 04:16:47.823106 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 04:16:47.823106 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 04:16:47.823106 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 04:16:47.823106 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 04:16:47.823106 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 04:16:47.823106 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 04:16:47.823106 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 04:16:47.823106 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 04:16:47.823106 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 04:16:47.823106 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 04:16:47.823106 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 88/189] 10.222.0.30 () {56 vars in 1096 bytes} [Tue Sep 6 04:16:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 04:16:47.828592 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee2e4e1-9079-4454-8e57-8e97ec0f8fa4 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:16:47.935765 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee2e4e1-9079-4454-8e57-8e97ec0f8fa4 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:16:47.972060 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130ca4f1-28d3-41bd-b0b0-984891ce1c3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:16:47.972678 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130ca4f1-28d3-41bd-b0b0-984891ce1c3a None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:16:47.972958 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130ca4f1-28d3-41bd-b0b0-984891ce1c3a None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:16:47.973232 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130ca4f1-28d3-41bd-b0b0-984891ce1c3a None admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:16:47.973979 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130ca4f1-28d3-41bd-b0b0-984891ce1c3a None admin] RBAC: Adding query filter params (name=admin) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:16:47.974280 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130ca4f1-28d3-41bd-b0b0-984891ce1c3a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:16:47.975822 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130ca4f1-28d3-41bd-b0b0-984891ce1c3a None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:16:47.984354 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 102/190] 10.222.0.30 () {56 vars in 1099 bytes} [Tue Sep 6 04:16:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:16:47.990701 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80410e96-697d-4cdd-bd2b-445a355907d1 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:16:48.004749 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130ca4f1-28d3-41bd-b0b0-984891ce1c3a None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:16:48.064975 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:16:48.110411 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80410e96-697d-4cdd-bd2b-445a355907d1 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:16:48.112169 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130ca4f1-28d3-41bd-b0b0-984891ce1c3a None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:16:48.123619 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:16:48.144311 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3173909-2e05-4426-846b-47b08cefa7a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:16:48.144985 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3173909-2e05-4426-846b-47b08cefa7a8 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:16:48.145261 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3173909-2e05-4426-846b-47b08cefa7a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:16:48.145559 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3173909-2e05-4426-846b-47b08cefa7a8 None admin] PATH_INFO: `/v3/users/2ac071cb56f741a6ab677be4e4baa126` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:16:48.148025 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafb3a45-cbaf-46d1-8ff1-26e74786958b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:16:48.148662 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafb3a45-cbaf-46d1-8ff1-26e74786958b None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:16:48.148941 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafb3a45-cbaf-46d1-8ff1-26e74786958b None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:16:48.149213 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafb3a45-cbaf-46d1-8ff1-26e74786958b None admin] PATH_INFO: `/v3/domains/default` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:16:48.150361 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eafb3a45-cbaf-46d1-8ff1-26e74786958b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:16:48.151811 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eafb3a45-cbaf-46d1-8ff1-26e74786958b None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:16:48.153415 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 103/191] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Sep 6 04:16:48 2022] GET /identity/v3/domains/default => generated 199 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 04:16:48.154181 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafb3a45-cbaf-46d1-8ff1-26e74786958b None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:16:48.163796 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3173909-2e05-4426-846b-47b08cefa7a8 None admin] RBAC: Authorizing `identity:get_user(user_id=2ac071cb56f741a6ab677be4e4baa126)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:16:48.165254 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3173909-2e05-4426-846b-47b08cefa7a8 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:16:48.166923 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 89/192] 10.222.0.30 () {56 vars in 1177 bytes} [Tue Sep 6 04:16:47 2022] GET /identity/v3/users/2ac071cb56f741a6ab677be4e4baa126 => generated 283 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:16:48.169584 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3173909-2e05-4426-846b-47b08cefa7a8 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:16:48.253790 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafb3a45-cbaf-46d1-8ff1-26e74786958b None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:16:48.265622 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3173909-2e05-4426-846b-47b08cefa7a8 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:16:48.276021 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:16:48.287583 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d268953-43cd-4d93-bd8a-d499d95de052 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:16:48.288261 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d268953-43cd-4d93-bd8a-d499d95de052 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:16:48.288536 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d268953-43cd-4d93-bd8a-d499d95de052 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:16:48.288807 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d268953-43cd-4d93-bd8a-d499d95de052 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:16:48.299067 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d268953-43cd-4d93-bd8a-d499d95de052 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 04:16:48.299996 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b854be0-2b83-4b66-94eb-869ab9745b66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:16:48.300625 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b854be0-2b83-4b66-94eb-869ab9745b66 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:16:48.300906 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b854be0-2b83-4b66-94eb-869ab9745b66 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:16:48.301177 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b854be0-2b83-4b66-94eb-869ab9745b66 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:16:48.304596 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-0d268953-43cd-4d93-bd8a-d499d95de052 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 06 04:16:48.304596 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 04:16:48.304596 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 04:16:48.304596 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 04:16:48.304596 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 04:16:48.304596 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 04:16:48.304596 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 04:16:48.304596 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 04:16:48.304596 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 04:16:48.304596 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 04:16:48.304596 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 04:16:48.304596 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 04:16:48.304596 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 04:16:48.304596 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 04:16:48.304596 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 04:16:48.304596 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 04:16:48.305519 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 04:16:48.305519 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 04:16:48.305519 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 04:16:48.305519 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 04:16:48.305519 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 04:16:48.305519 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 04:16:48.305519 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 04:16:48.305519 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 04:16:48.305519 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 04:16:48.305519 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 04:16:48.305519 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 04:16:48.305519 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 04:16:48.305519 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 04:16:48.305519 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 04:16:48.305519 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 04:16:48.305519 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 04:16:48.306333 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 04:16:48.306333 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 04:16:48.306333 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 04:16:48.306333 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 04:16:48.306333 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 04:16:48.306333 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 04:16:48.306333 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 04:16:48.306333 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 04:16:48.306333 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 04:16:48.306333 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 04:16:48.306333 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 06 04:16:48.306333 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 04:16:48.306333 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 104/193] 10.222.0.30 () {56 vars in 1114 bytes} [Tue Sep 6 04:16:48 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 04:16:48.306981 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d268953-43cd-4d93-bd8a-d499d95de052 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:16:48.314802 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b854be0-2b83-4b66-94eb-869ab9745b66 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 04:16:48.319703 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-6b854be0-2b83-4b66-94eb-869ab9745b66 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 06 04:16:48.319703 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 04:16:48.319703 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 04:16:48.319703 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 04:16:48.319703 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 04:16:48.319703 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 04:16:48.319703 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 04:16:48.319703 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 04:16:48.319703 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 04:16:48.319703 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 04:16:48.319703 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 04:16:48.319703 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 04:16:48.319703 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 04:16:48.319703 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 04:16:48.319703 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 04:16:48.319703 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 04:16:48.320620 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 04:16:48.320620 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 04:16:48.320620 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 04:16:48.320620 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 04:16:48.320620 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 04:16:48.320620 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 04:16:48.320620 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 04:16:48.320620 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 04:16:48.320620 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 04:16:48.320620 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 04:16:48.320620 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 04:16:48.320620 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 04:16:48.320620 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 04:16:48.320620 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 04:16:48.320620 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 04:16:48.320620 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 04:16:48.321452 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 04:16:48.321452 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 04:16:48.321452 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 04:16:48.321452 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 04:16:48.321452 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 04:16:48.321452 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 04:16:48.321452 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 04:16:48.321452 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 04:16:48.321452 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 04:16:48.321452 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 04:16:48.321452 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 06 04:16:48.321452 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 04:16:48.321452 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 90/194] 10.222.0.30 () {56 vars in 1099 bytes} [Tue Sep 6 04:16:48 2022] GET /identity/v3/roles/reader => generated 84 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 04:16:48.322106 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b854be0-2b83-4b66-94eb-869ab9745b66 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:16:48.351581 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:16:48.409653 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d268953-43cd-4d93-bd8a-d499d95de052 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:16:48.424702 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b854be0-2b83-4b66-94eb-869ab9745b66 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:16:48.453963 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0dbc2f-091c-4c8d-aee7-7cf514c1708b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:16:48.454579 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0dbc2f-091c-4c8d-aee7-7cf514c1708b None admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:16:48.454855 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0dbc2f-091c-4c8d-aee7-7cf514c1708b None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:16:48.455124 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0dbc2f-091c-4c8d-aee7-7cf514c1708b None admin] PATH_INFO: `/v3/users` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:16:48.455785 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0dbc2f-091c-4c8d-aee7-7cf514c1708b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:16:48.457230 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0dbc2f-091c-4c8d-aee7-7cf514c1708b None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:16:48.464467 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b4b486-54ae-4ae0-94cd-c66bf2642a00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:16:48.465093 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b4b486-54ae-4ae0-94cd-c66bf2642a00 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:16:48.465415 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b4b486-54ae-4ae0-94cd-c66bf2642a00 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:16:48.465693 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b4b486-54ae-4ae0-94cd-c66bf2642a00 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:16:48.534143 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b4b486-54ae-4ae0-94cd-c66bf2642a00 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:16:48.535706 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b4b486-54ae-4ae0-94cd-c66bf2642a00 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:16:48.551409 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 105/195] 10.222.0.30 () {60 vars in 1132 bytes} [Tue Sep 6 04:16:48 2022] POST /identity/v3/users => generated 297 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 04:16:48.552237 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0dbc2f-091c-4c8d-aee7-7cf514c1708b None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:16:48.570851 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:16:48.612030 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:16:48.645100 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 91/196] 10.222.0.30 () {58 vars in 1282 bytes} [Tue Sep 6 04:16:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 06 04:16:48.645929 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b4b486-54ae-4ae0-94cd-c66bf2642a00 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:16:48.690027 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0dbc2f-091c-4c8d-aee7-7cf514c1708b None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:16:48.731617 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a69296-a64c-4f8d-bef5-fff2f303cbb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:16:48.732246 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a69296-a64c-4f8d-bef5-fff2f303cbb6 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:16:48.732549 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a69296-a64c-4f8d-bef5-fff2f303cbb6 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:16:48.732817 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a69296-a64c-4f8d-bef5-fff2f303cbb6 None admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:16:48.733531 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a69296-a64c-4f8d-bef5-fff2f303cbb6 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:16:48.733857 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a69296-a64c-4f8d-bef5-fff2f303cbb6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:16:48.735253 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a69296-a64c-4f8d-bef5-fff2f303cbb6 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:16:48.744233 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 106/197] 10.222.0.30 () {56 vars in 1111 bytes} [Tue Sep 6 04:16:48 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:16:48.744930 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a69296-a64c-4f8d-bef5-fff2f303cbb6 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:16:48.769610 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b4b486-54ae-4ae0-94cd-c66bf2642a00 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:16:48.815638 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:16:48.830536 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ed793c-a6d8-4d02-81f4-27381d10d4a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:16:48.831237 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ed793c-a6d8-4d02-81f4-27381d10d4a1 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:16:48.831516 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ed793c-a6d8-4d02-81f4-27381d10d4a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:16:48.831787 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ed793c-a6d8-4d02-81f4-27381d10d4a1 None admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:16:48.832475 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ed793c-a6d8-4d02-81f4-27381d10d4a1 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:16:48.832775 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ed793c-a6d8-4d02-81f4-27381d10d4a1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:16:48.834445 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ed793c-a6d8-4d02-81f4-27381d10d4a1 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:16:48.846838 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 92/198] 10.222.0.30 () {56 vars in 1101 bytes} [Tue Sep 6 04:16:48 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:16:48.854245 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ed793c-a6d8-4d02-81f4-27381d10d4a1 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:16:48.884021 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:16:48.888496 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a69296-a64c-4f8d-bef5-fff2f303cbb6 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:16:48.942468 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f3d9c5-e863-4c0b-a4c5-9c7b78b6a4ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:16:48.943027 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f3d9c5-e863-4c0b-a4c5-9c7b78b6a4ac None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:16:48.943916 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f3d9c5-e863-4c0b-a4c5-9c7b78b6a4ac None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:16:48.944137 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f3d9c5-e863-4c0b-a4c5-9c7b78b6a4ac None admin] PATH_INFO: `/v3/projects/b57372dfc633444d9f127657ca57d59c` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:16:48.946304 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f3d9c5-e863-4c0b-a4c5-9c7b78b6a4ac None admin] RBAC: Authorizing `identity:get_project(project_id=b57372dfc633444d9f127657ca57d59c)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:16:48.949166 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f3d9c5-e863-4c0b-a4c5-9c7b78b6a4ac None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:16:48.951124 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 107/199] 10.222.0.30 () {56 vars in 1186 bytes} [Tue Sep 6 04:16:48 2022] GET /identity/v3/projects/b57372dfc633444d9f127657ca57d59c => generated 302 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:16:48.952107 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f3d9c5-e863-4c0b-a4c5-9c7b78b6a4ac None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:16:48.993637 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ed793c-a6d8-4d02-81f4-27381d10d4a1 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:16:49.038178 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c48ac7b-0514-4e00-bb22-0feeb1dabafe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:16:49.039149 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c48ac7b-0514-4e00-bb22-0feeb1dabafe None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:16:49.039725 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c48ac7b-0514-4e00-bb22-0feeb1dabafe None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:16:49.040663 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c48ac7b-0514-4e00-bb22-0feeb1dabafe None admin] PATH_INFO: `/v3/users/2ac071cb56f741a6ab677be4e4baa126` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:16:49.042033 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c48ac7b-0514-4e00-bb22-0feeb1dabafe None admin] RBAC: Authorizing `identity:get_user(user_id=2ac071cb56f741a6ab677be4e4baa126)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:16:49.044314 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c48ac7b-0514-4e00-bb22-0feeb1dabafe None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:16:49.046425 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 93/200] 10.222.0.30 () {56 vars in 1177 bytes} [Tue Sep 6 04:16:48 2022] GET /identity/v3/users/2ac071cb56f741a6ab677be4e4baa126 => generated 283 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:16:49.048003 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c48ac7b-0514-4e00-bb22-0feeb1dabafe None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:16:49.066692 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f3d9c5-e863-4c0b-a4c5-9c7b78b6a4ac None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:16:49.108706 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de94f127-b997-408b-88af-5b2bb2b11004 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:16:49.110168 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de94f127-b997-408b-88af-5b2bb2b11004 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:16:49.110883 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de94f127-b997-408b-88af-5b2bb2b11004 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:16:49.111330 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de94f127-b997-408b-88af-5b2bb2b11004 None admin] PATH_INFO: `/v3/users/501af79e7c0d4d768256c193a0305538` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:16:49.112932 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de94f127-b997-408b-88af-5b2bb2b11004 None admin] RBAC: Authorizing `identity:get_user(user_id=501af79e7c0d4d768256c193a0305538)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:16:49.115052 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de94f127-b997-408b-88af-5b2bb2b11004 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:16:49.117627 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 108/201] 10.222.0.30 () {56 vars in 1177 bytes} [Tue Sep 6 04:16:48 2022] GET /identity/v3/users/501af79e7c0d4d768256c193a0305538 => generated 289 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:16:49.118878 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de94f127-b997-408b-88af-5b2bb2b11004 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:16:49.123944 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:16:49.169107 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:16:49.180874 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c48ac7b-0514-4e00-bb22-0feeb1dabafe None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:16:49.230070 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d992e7-372b-46ec-b115-ded6beb62c7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:16:49.231373 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d992e7-372b-46ec-b115-ded6beb62c7b None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:16:49.231810 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d992e7-372b-46ec-b115-ded6beb62c7b None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:16:49.232231 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d992e7-372b-46ec-b115-ded6beb62c7b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:16:49.233783 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d992e7-372b-46ec-b115-ded6beb62c7b None admin] RBAC: Adding query filter params (role.id=ea4c814713b149108e488de855b025e4, scope.project.id=b57372dfc633444d9f127657ca57d59c, user.id=2ac071cb56f741a6ab677be4e4baa126) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:16:49.234246 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d992e7-372b-46ec-b115-ded6beb62c7b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:16:49.236413 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d992e7-372b-46ec-b115-ded6beb62c7b None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:16:49.243944 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 94/202] 10.222.0.30 () {56 vars in 1374 bytes} [Tue Sep 6 04:16:49 2022] GET /identity/v3/role_assignments?user.id=2ac071cb56f741a6ab677be4e4baa126&scope.project.id=b57372dfc633444d9f127657ca57d59c&role.id=ea4c814713b149108e488de855b025e4 => generated 259 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:16:49.244933 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d992e7-372b-46ec-b115-ded6beb62c7b None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:16:49.264758 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de94f127-b997-408b-88af-5b2bb2b11004 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:16:49.322141 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a84d7b-0993-419e-b129-ac0b66d11c21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:16:49.322798 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a84d7b-0993-419e-b129-ac0b66d11c21 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:16:49.323060 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a84d7b-0993-419e-b129-ac0b66d11c21 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:16:49.323339 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a84d7b-0993-419e-b129-ac0b66d11c21 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:16:49.445091 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d992e7-372b-46ec-b115-ded6beb62c7b None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:16:49.462996 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a84d7b-0993-419e-b129-ac0b66d11c21 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:16:49.466687 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a84d7b-0993-419e-b129-ac0b66d11c21 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:16:49.485064 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:16:49.510581 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b949509e-8272-4294-829c-bd869e3d8ece None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:16:49.511211 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b949509e-8272-4294-829c-bd869e3d8ece None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:16:49.511488 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b949509e-8272-4294-829c-bd869e3d8ece None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:16:49.511759 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b949509e-8272-4294-829c-bd869e3d8ece None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:16:49.511955 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:16:49.583330 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 109/203] 10.222.0.30 () {58 vars in 1282 bytes} [Tue Sep 6 04:16:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 465 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 06 04:16:49.589563 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a84d7b-0993-419e-b129-ac0b66d11c21 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:16:49.607475 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b949509e-8272-4294-829c-bd869e3d8ece None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:16:49.608968 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b949509e-8272-4294-829c-bd869e3d8ece None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:16:49.710184 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 95/204] 10.222.0.30 () {58 vars in 1282 bytes} [Tue Sep 6 04:16:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 465 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 06 04:16:49.722333 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b949509e-8272-4294-829c-bd869e3d8ece None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:16:49.754170 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a84d7b-0993-419e-b129-ac0b66d11c21 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:16:49.770086 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:16:49.793599 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f546f39e-9781-45b0-946c-aefd8ad8c2b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:16:49.794491 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f546f39e-9781-45b0-946c-aefd8ad8c2b4 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:16:49.795009 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f546f39e-9781-45b0-946c-aefd8ad8c2b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:16:49.795441 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f546f39e-9781-45b0-946c-aefd8ad8c2b4 None admin] PATH_INFO: `/v3/projects/b57372dfc633444d9f127657ca57d59c` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:16:49.800530 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f546f39e-9781-45b0-946c-aefd8ad8c2b4 None admin] RBAC: Authorizing `identity:get_project(project_id=b57372dfc633444d9f127657ca57d59c)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:16:49.802287 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f546f39e-9781-45b0-946c-aefd8ad8c2b4 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:16:49.804729 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 110/205] 10.222.0.30 () {56 vars in 1186 bytes} [Tue Sep 6 04:16:49 2022] GET /identity/v3/projects/b57372dfc633444d9f127657ca57d59c => generated 302 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 04:16:49.814375 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f546f39e-9781-45b0-946c-aefd8ad8c2b4 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:16:49.823430 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:16:49.895481 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b949509e-8272-4294-829c-bd869e3d8ece None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:16:49.968243 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeef236a-1ade-4010-81e3-09ed545b1945 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:16:49.968885 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeef236a-1ade-4010-81e3-09ed545b1945 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:16:49.969164 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeef236a-1ade-4010-81e3-09ed545b1945 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:16:49.969465 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeef236a-1ade-4010-81e3-09ed545b1945 None admin] PATH_INFO: `/v3/projects/a88c6977d0554a1eab3e30118ede2f46` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:16:49.970957 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeef236a-1ade-4010-81e3-09ed545b1945 None admin] RBAC: Authorizing `identity:get_project(project_id=a88c6977d0554a1eab3e30118ede2f46)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:16:49.972397 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeef236a-1ade-4010-81e3-09ed545b1945 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:16:49.974780 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 96/206] 10.222.0.30 () {56 vars in 1186 bytes} [Tue Sep 6 04:16:49 2022] GET /identity/v3/projects/a88c6977d0554a1eab3e30118ede2f46 => generated 298 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:16:49.994612 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeef236a-1ade-4010-81e3-09ed545b1945 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:16:50.005096 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f546f39e-9781-45b0-946c-aefd8ad8c2b4 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:16:50.075803 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9182f078-0a7e-49f4-a348-25f421f45261 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:16:50.076420 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9182f078-0a7e-49f4-a348-25f421f45261 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:16:50.076698 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9182f078-0a7e-49f4-a348-25f421f45261 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:16:50.076968 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9182f078-0a7e-49f4-a348-25f421f45261 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:16:50.077736 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9182f078-0a7e-49f4-a348-25f421f45261 None admin] RBAC: Adding query filter params (role.id=77197446dd2541b9a4fab9a654c08fc9, scope.project.id=b57372dfc633444d9f127657ca57d59c, user.id=2ac071cb56f741a6ab677be4e4baa126) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:16:50.078045 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9182f078-0a7e-49f4-a348-25f421f45261 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:16:50.079499 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9182f078-0a7e-49f4-a348-25f421f45261 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:16:50.094756 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 111/207] 10.222.0.30 () {56 vars in 1374 bytes} [Tue Sep 6 04:16:49 2022] GET /identity/v3/role_assignments?user.id=2ac071cb56f741a6ab677be4e4baa126&scope.project.id=b57372dfc633444d9f127657ca57d59c&role.id=77197446dd2541b9a4fab9a654c08fc9 => generated 259 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:16:50.154972 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:16:50.170730 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeef236a-1ade-4010-81e3-09ed545b1945 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:16:50.221338 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f001cf83-a50b-48a0-86f2-a342a501bf07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:16:50.221976 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f001cf83-a50b-48a0-86f2-a342a501bf07 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:16:50.222260 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f001cf83-a50b-48a0-86f2-a342a501bf07 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:16:50.222529 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f001cf83-a50b-48a0-86f2-a342a501bf07 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:16:50.223276 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f001cf83-a50b-48a0-86f2-a342a501bf07 None admin] RBAC: Adding query filter params (role.id=3c011524d1df425b902b8b32c1b485c3, scope.project.id=a88c6977d0554a1eab3e30118ede2f46, user.id=501af79e7c0d4d768256c193a0305538) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:16:50.223580 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f001cf83-a50b-48a0-86f2-a342a501bf07 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:16:50.225227 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f001cf83-a50b-48a0-86f2-a342a501bf07 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:16:50.234179 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 97/208] 10.222.0.30 () {56 vars in 1374 bytes} [Tue Sep 6 04:16:49 2022] GET /identity/v3/role_assignments?user.id=501af79e7c0d4d768256c193a0305538&scope.project.id=a88c6977d0554a1eab3e30118ede2f46&role.id=3c011524d1df425b902b8b32c1b485c3 => generated 608 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 04:16:50.878889 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdaded6-0c96-4efd-8ada-512cfae4bbcf None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:16:50.879396 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdaded6-0c96-4efd-8ada-512cfae4bbcf None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:16:50.879501 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdaded6-0c96-4efd-8ada-512cfae4bbcf None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:16:50.881020 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 112/209] 10.222.0.30 () {54 vars in 927 bytes} [Tue Sep 6 04:16:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 04:16:50.886608 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1eb6e3-45fd-46ea-ae2b-fcafe6e2b11b None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:16:50.887024 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1eb6e3-45fd-46ea-ae2b-fcafe6e2b11b None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:16:50.887217 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1eb6e3-45fd-46ea-ae2b-fcafe6e2b11b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:16:50.908756 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-cc1eb6e3-45fd-46ea-ae2b-fcafe6e2b11b None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:16:50.909953 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-cc1eb6e3-45fd-46ea-ae2b-fcafe6e2b11b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:16:50.938480 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cff21c3-26e4-48b2-9371-7673f54de302 None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:16:50.938785 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cff21c3-26e4-48b2-9371-7673f54de302 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:16:50.939142 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cff21c3-26e4-48b2-9371-7673f54de302 None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:16:50.940609 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 113/210] 10.222.0.30 () {54 vars in 927 bytes} [Tue Sep 6 04:16:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 06 04:16:50.946928 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06141db3-570b-4f58-8365-893641d5ad4f None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:16:50.947324 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06141db3-570b-4f58-8365-893641d5ad4f None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:16:50.947699 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06141db3-570b-4f58-8365-893641d5ad4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:16:50.970974 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-06141db3-570b-4f58-8365-893641d5ad4f None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:16:50.972578 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-06141db3-570b-4f58-8365-893641d5ad4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:16:50.978977 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 98/211] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Sep 6 04:16:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 04:16:51.041536 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:16:51.046992 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 114/212] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Sep 6 04:16:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 04:16:51.256234 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1eb6e3-45fd-46ea-ae2b-fcafe6e2b11b None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:16:51.323492 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06141db3-570b-4f58-8365-893641d5ad4f None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:16:51.341333 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:16:51.359042 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1eb6e3-45fd-46ea-ae2b-fcafe6e2b11b None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:16:51.410137 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f734b0-f5b8-431f-9025-51889f63ca21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:16:51.411377 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f734b0-f5b8-431f-9025-51889f63ca21 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:16:51.412228 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f734b0-f5b8-431f-9025-51889f63ca21 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:16:51.412576 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f734b0-f5b8-431f-9025-51889f63ca21 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:16:51.427005 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f734b0-f5b8-431f-9025-51889f63ca21 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 04:16:51.435126 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-f8f734b0-f5b8-431f-9025-51889f63ca21 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 06 04:16:51.435126 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 04:16:51.435126 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 04:16:51.435126 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 04:16:51.435126 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 04:16:51.435126 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 04:16:51.435126 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 04:16:51.435126 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 04:16:51.435126 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 04:16:51.435126 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 04:16:51.435126 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 04:16:51.435126 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 04:16:51.435126 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 04:16:51.435126 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 04:16:51.435126 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 04:16:51.435126 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 04:16:51.436672 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 04:16:51.436672 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 04:16:51.436672 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 04:16:51.436672 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 04:16:51.436672 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 04:16:51.436672 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 04:16:51.436672 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 04:16:51.436672 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 04:16:51.436672 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 04:16:51.436672 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 04:16:51.436672 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 04:16:51.436672 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 04:16:51.436672 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 04:16:51.436672 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 04:16:51.436672 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 04:16:51.436672 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 04:16:51.437481 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 04:16:51.437481 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 04:16:51.437481 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 04:16:51.437481 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 04:16:51.437481 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 04:16:51.437481 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 04:16:51.437481 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 04:16:51.437481 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 04:16:51.437481 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 04:16:51.437481 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 04:16:51.437481 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 06 04:16:51.437481 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 04:16:51.437481 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 99/213] 10.222.0.30 () {56 vars in 1099 bytes} [Tue Sep 6 04:16:51 2022] GET /identity/v3/roles/member => generated 84 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 04:16:51.448873 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f734b0-f5b8-431f-9025-51889f63ca21 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:16:51.456313 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06141db3-570b-4f58-8365-893641d5ad4f None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:16:51.493477 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8bbe604-9993-40c4-9d36-3681396bfc7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:16:51.494315 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bbe604-9993-40c4-9d36-3681396bfc7b None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:16:51.494640 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bbe604-9993-40c4-9d36-3681396bfc7b None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:16:51.494919 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bbe604-9993-40c4-9d36-3681396bfc7b None admin] PATH_INFO: `/v3/domains/default` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:16:51.496053 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8bbe604-9993-40c4-9d36-3681396bfc7b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:16:51.497554 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8bbe604-9993-40c4-9d36-3681396bfc7b None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:16:51.499332 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 115/214] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Sep 6 04:16:51 2022] GET /identity/v3/domains/default => generated 199 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:16:51.501524 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1fd761-5f46-481d-8e1d-8eb187f2307e None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:16:51.501806 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1fd761-5f46-481d-8e1d-8eb187f2307e None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:16:51.502090 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1fd761-5f46-481d-8e1d-8eb187f2307e None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:16:51.503839 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 116/215] 10.222.0.30 () {54 vars in 927 bytes} [Tue Sep 6 04:16:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 04:16:51.509007 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1fd761-5f46-481d-8e1d-8eb187f2307e None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:16:51.576783 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f734b0-f5b8-431f-9025-51889f63ca21 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:16:51.581597 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:16:51.612851 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d547194a-1432-4a35-a5c4-a879b9b5e3a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:16:51.613498 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d547194a-1432-4a35-a5c4-a879b9b5e3a7 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:16:51.613777 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d547194a-1432-4a35-a5c4-a879b9b5e3a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:16:51.614045 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d547194a-1432-4a35-a5c4-a879b9b5e3a7 None admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:16:51.614711 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d547194a-1432-4a35-a5c4-a879b9b5e3a7 None admin] RBAC: Adding query filter params (name=member) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:16:51.615681 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d547194a-1432-4a35-a5c4-a879b9b5e3a7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:16:51.617203 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d547194a-1432-4a35-a5c4-a879b9b5e3a7 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:16:51.618554 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1fd761-5f46-481d-8e1d-8eb187f2307e None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:16:51.625584 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 100/216] 10.222.0.30 () {56 vars in 1101 bytes} [Tue Sep 6 04:16:51 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 04:16:51.626749 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc30bd43-c1c4-4719-be2e-99beb1b61295 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:16:51.627032 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc30bd43-c1c4-4719-be2e-99beb1b61295 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:16:51.627554 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc30bd43-c1c4-4719-be2e-99beb1b61295 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:16:51.646791 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-dc30bd43-c1c4-4719-be2e-99beb1b61295 None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:16:51.647972 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-dc30bd43-c1c4-4719-be2e-99beb1b61295 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:16:51.650061 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:16:51.658708 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0426577b-4644-4900-8490-86de6a0cd1a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:16:51.659402 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0426577b-4644-4900-8490-86de6a0cd1a5 None admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:16:51.659667 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0426577b-4644-4900-8490-86de6a0cd1a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:16:51.659933 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0426577b-4644-4900-8490-86de6a0cd1a5 None admin] PATH_INFO: `/v3/users` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:16:51.661210 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0426577b-4644-4900-8490-86de6a0cd1a5 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:16:51.662155 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0426577b-4644-4900-8490-86de6a0cd1a5 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:16:51.715842 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 101/217] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Sep 6 04:16:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 04:16:51.716739 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc30bd43-c1c4-4719-be2e-99beb1b61295 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:16:51.746386 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 117/218] 10.222.0.30 () {60 vars in 1132 bytes} [Tue Sep 6 04:16:51 2022] POST /identity/v3/users => generated 297 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 04:16:51.844142 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc30bd43-c1c4-4719-be2e-99beb1b61295 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:16:51.885141 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b853d525-eba7-4bb4-9877-13cdade0e3d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:16:51.886103 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b853d525-eba7-4bb4-9877-13cdade0e3d0 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:16:51.886536 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b853d525-eba7-4bb4-9877-13cdade0e3d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:16:51.886812 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b853d525-eba7-4bb4-9877-13cdade0e3d0 None admin] PATH_INFO: `/v3/users/35ab231490514d90a4c3fcb3d63e63bd` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:16:51.895381 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:16:51.902866 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b853d525-eba7-4bb4-9877-13cdade0e3d0 None admin] RBAC: Authorizing `identity:get_user(user_id=35ab231490514d90a4c3fcb3d63e63bd)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:16:51.909334 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b853d525-eba7-4bb4-9877-13cdade0e3d0 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:16:51.909334 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 102/219] 10.222.0.30 () {56 vars in 1177 bytes} [Tue Sep 6 04:16:51 2022] GET /identity/v3/users/35ab231490514d90a4c3fcb3d63e63bd => generated 297 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:16:51.912974 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0426577b-4644-4900-8490-86de6a0cd1a5 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:16:51.992775 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b853d525-eba7-4bb4-9877-13cdade0e3d0 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:16:52.033508 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0426577b-4644-4900-8490-86de6a0cd1a5 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:16:52.065704 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:16:52.084807 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54d5658-97c9-4c96-b289-8bce98566231 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:16:52.088324 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54d5658-97c9-4c96-b289-8bce98566231 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:16:52.088324 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54d5658-97c9-4c96-b289-8bce98566231 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:16:52.088324 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54d5658-97c9-4c96-b289-8bce98566231 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:16:52.132332 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b853d525-eba7-4bb4-9877-13cdade0e3d0 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:16:52.170521 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54d5658-97c9-4c96-b289-8bce98566231 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:16:52.172611 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54d5658-97c9-4c96-b289-8bce98566231 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:16:52.181214 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97321e69-4c21-432c-be89-446e43f95790 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:16:52.182764 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97321e69-4c21-432c-be89-446e43f95790 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:16:52.183264 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97321e69-4c21-432c-be89-446e43f95790 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:16:52.183752 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97321e69-4c21-432c-be89-446e43f95790 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:16:52.200369 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97321e69-4c21-432c-be89-446e43f95790 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 04:16:52.207581 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-97321e69-4c21-432c-be89-446e43f95790 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 04:16:52.207581 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 04:16:52.207581 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 04:16:52.207581 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 04:16:52.207581 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 04:16:52.207581 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 04:16:52.207581 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 04:16:52.207581 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 04:16:52.207581 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 04:16:52.207581 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 04:16:52.207581 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 04:16:52.207581 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 04:16:52.207581 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 04:16:52.207581 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 04:16:52.207581 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 04:16:52.207581 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 04:16:52.208771 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 04:16:52.208771 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 04:16:52.208771 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 04:16:52.208771 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 04:16:52.208771 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 04:16:52.208771 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 04:16:52.208771 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 04:16:52.208771 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 04:16:52.208771 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 04:16:52.208771 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 04:16:52.208771 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 04:16:52.208771 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 04:16:52.208771 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 04:16:52.208771 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 04:16:52.208771 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 04:16:52.208771 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 04:16:52.209970 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 04:16:52.209970 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 04:16:52.209970 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 04:16:52.209970 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 04:16:52.209970 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 04:16:52.209970 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 04:16:52.209970 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 04:16:52.209970 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 04:16:52.209970 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 04:16:52.209970 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 04:16:52.209970 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 04:16:52.209970 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 04:16:52.209970 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 103/220] 10.222.0.30 () {56 vars in 1096 bytes} [Tue Sep 6 04:16:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 04:16:52.215352 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97321e69-4c21-432c-be89-446e43f95790 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:16:52.292138 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:16:52.303556 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 118/221] 10.222.0.30 () {58 vars in 1282 bytes} [Tue Sep 6 04:16:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 391 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 06 04:16:52.304590 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f84dce8-0ed1-4f3d-85dd-9656a1a3e4f9 None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:16:52.304878 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f84dce8-0ed1-4f3d-85dd-9656a1a3e4f9 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:16:52.305155 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f84dce8-0ed1-4f3d-85dd-9656a1a3e4f9 None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:16:52.306608 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 119/222] 10.222.0.30 () {54 vars in 927 bytes} [Tue Sep 6 04:16:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 04:16:52.308840 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f84dce8-0ed1-4f3d-85dd-9656a1a3e4f9 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:16:52.334468 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:16:52.356478 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97321e69-4c21-432c-be89-446e43f95790 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:16:52.397175 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6f2e5b-3e62-4d56-ba87-b7924720dbd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:16:52.398134 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6f2e5b-3e62-4d56-ba87-b7924720dbd9 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:16:52.402960 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6f2e5b-3e62-4d56-ba87-b7924720dbd9 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:16:52.402960 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6f2e5b-3e62-4d56-ba87-b7924720dbd9 None admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:16:52.402960 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6f2e5b-3e62-4d56-ba87-b7924720dbd9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:16:52.402960 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6f2e5b-3e62-4d56-ba87-b7924720dbd9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:16:52.402960 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6f2e5b-3e62-4d56-ba87-b7924720dbd9 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:16:52.412101 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 104/223] 10.222.0.30 () {56 vars in 1099 bytes} [Tue Sep 6 04:16:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:16:52.413825 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28996f73-4cba-4627-901c-8053cae14b7e None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:16:52.414451 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28996f73-4cba-4627-901c-8053cae14b7e None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:16:52.414978 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28996f73-4cba-4627-901c-8053cae14b7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:16:52.429903 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f84dce8-0ed1-4f3d-85dd-9656a1a3e4f9 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:16:52.436090 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-28996f73-4cba-4627-901c-8053cae14b7e None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:16:52.437950 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-28996f73-4cba-4627-901c-8053cae14b7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:16:52.466662 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8d09cd-ec12-4a50-aa53-42ed79b7d8bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:16:52.467307 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8d09cd-ec12-4a50-aa53-42ed79b7d8bc None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:16:52.467584 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8d09cd-ec12-4a50-aa53-42ed79b7d8bc None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:16:52.467855 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8d09cd-ec12-4a50-aa53-42ed79b7d8bc None admin] PATH_INFO: `/v3/projects/b57372dfc633444d9f127657ca57d59c` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:16:52.468996 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b8d09cd-ec12-4a50-aa53-42ed79b7d8bc None admin] RBAC: Authorizing `identity:get_project(project_id=b57372dfc633444d9f127657ca57d59c)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:16:52.470856 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b8d09cd-ec12-4a50-aa53-42ed79b7d8bc None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:16:52.472329 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 120/224] 10.222.0.30 () {56 vars in 1186 bytes} [Tue Sep 6 04:16:52 2022] GET /identity/v3/projects/b57372dfc633444d9f127657ca57d59c => generated 302 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:16:52.473078 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8d09cd-ec12-4a50-aa53-42ed79b7d8bc None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:16:52.539191 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 105/225] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Sep 6 04:16:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 04:16:52.539191 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28996f73-4cba-4627-901c-8053cae14b7e None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:16:52.602848 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:16:52.612756 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8d09cd-ec12-4a50-aa53-42ed79b7d8bc None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:16:52.652680 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:16:52.681195 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1718f0fc-7407-4a0a-8a62-3784a3fa95c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:16:52.682999 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1718f0fc-7407-4a0a-8a62-3784a3fa95c7 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:16:52.683286 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1718f0fc-7407-4a0a-8a62-3784a3fa95c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:16:52.683557 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1718f0fc-7407-4a0a-8a62-3784a3fa95c7 None admin] PATH_INFO: `/v3/users/2ac071cb56f741a6ab677be4e4baa126` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:16:52.685572 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1718f0fc-7407-4a0a-8a62-3784a3fa95c7 None admin] RBAC: Authorizing `identity:get_user(user_id=2ac071cb56f741a6ab677be4e4baa126)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:16:52.687036 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1718f0fc-7407-4a0a-8a62-3784a3fa95c7 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:16:52.688644 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 121/226] 10.222.0.30 () {56 vars in 1177 bytes} [Tue Sep 6 04:16:52 2022] GET /identity/v3/users/2ac071cb56f741a6ab677be4e4baa126 => generated 283 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:16:52.701004 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1718f0fc-7407-4a0a-8a62-3784a3fa95c7 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:16:52.750113 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28996f73-4cba-4627-901c-8053cae14b7e None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:16:52.801946 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7519bc-249e-4dad-a147-09ed4ce5a41b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:16:52.802633 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7519bc-249e-4dad-a147-09ed4ce5a41b None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:16:52.802916 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7519bc-249e-4dad-a147-09ed4ce5a41b None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:16:52.803281 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7519bc-249e-4dad-a147-09ed4ce5a41b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:16:52.804027 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7519bc-249e-4dad-a147-09ed4ce5a41b None admin] RBAC: Adding query filter params (role.id=29607b2de39f40d68bb6ceef64b8a732, scope.project.id=b57372dfc633444d9f127657ca57d59c, user.id=35ab231490514d90a4c3fcb3d63e63bd) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:16:52.804338 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7519bc-249e-4dad-a147-09ed4ce5a41b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:16:52.809508 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7519bc-249e-4dad-a147-09ed4ce5a41b None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:16:52.818929 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 106/227] 10.222.0.30 () {56 vars in 1374 bytes} [Tue Sep 6 04:16:52 2022] GET /identity/v3/role_assignments?user.id=35ab231490514d90a4c3fcb3d63e63bd&scope.project.id=b57372dfc633444d9f127657ca57d59c&role.id=29607b2de39f40d68bb6ceef64b8a732 => generated 259 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:16:52.896433 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1718f0fc-7407-4a0a-8a62-3784a3fa95c7 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:16:52.935541 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7519bc-249e-4dad-a147-09ed4ce5a41b None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:16:52.961525 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0d42c1-a1fd-4fd7-a964-73bca16dd72a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:16:52.962265 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0d42c1-a1fd-4fd7-a964-73bca16dd72a None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:16:52.962549 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0d42c1-a1fd-4fd7-a964-73bca16dd72a None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:16:52.962825 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0d42c1-a1fd-4fd7-a964-73bca16dd72a None admin] PATH_INFO: `/v3/projects/b57372dfc633444d9f127657ca57d59c` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:16:52.964119 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0d42c1-a1fd-4fd7-a964-73bca16dd72a None admin] RBAC: Authorizing `identity:get_project(project_id=b57372dfc633444d9f127657ca57d59c)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:16:52.965624 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0d42c1-a1fd-4fd7-a964-73bca16dd72a None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:16:52.967324 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 122/228] 10.222.0.30 () {56 vars in 1186 bytes} [Tue Sep 6 04:16:52 2022] GET /identity/v3/projects/b57372dfc633444d9f127657ca57d59c => generated 302 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:16:52.973190 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0d42c1-a1fd-4fd7-a964-73bca16dd72a None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:16:53.002239 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:16:53.081333 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7519bc-249e-4dad-a147-09ed4ce5a41b None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:16:53.089993 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:16:53.111740 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16648aac-b75c-4f14-bbca-b64a074ba7d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:16:53.112737 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16648aac-b75c-4f14-bbca-b64a074ba7d6 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:16:53.113170 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16648aac-b75c-4f14-bbca-b64a074ba7d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:16:53.113643 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16648aac-b75c-4f14-bbca-b64a074ba7d6 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:16:53.126864 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16648aac-b75c-4f14-bbca-b64a074ba7d6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 04:16:53.132965 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-16648aac-b75c-4f14-bbca-b64a074ba7d6 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 06 04:16:53.132965 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 04:16:53.132965 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 04:16:53.132965 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 04:16:53.132965 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 04:16:53.132965 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 04:16:53.132965 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 04:16:53.132965 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 04:16:53.132965 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 04:16:53.132965 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 04:16:53.132965 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 04:16:53.132965 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 04:16:53.132965 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 04:16:53.132965 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 04:16:53.132965 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 04:16:53.132965 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 04:16:53.133933 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 04:16:53.133933 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 04:16:53.133933 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 04:16:53.133933 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 04:16:53.133933 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 04:16:53.133933 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 04:16:53.133933 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 04:16:53.133933 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 04:16:53.133933 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 04:16:53.133933 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 04:16:53.133933 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 04:16:53.133933 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 04:16:53.133933 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 04:16:53.133933 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 04:16:53.133933 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 04:16:53.133933 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 04:16:53.134722 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 04:16:53.134722 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 04:16:53.134722 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 04:16:53.134722 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 04:16:53.134722 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 04:16:53.134722 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 04:16:53.134722 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 04:16:53.134722 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 04:16:53.134722 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 04:16:53.134722 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 04:16:53.134722 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 06 04:16:53.134722 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 04:16:53.137042 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 107/229] 10.222.0.30 () {56 vars in 1114 bytes} [Tue Sep 6 04:16:52 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 04:16:53.142778 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16648aac-b75c-4f14-bbca-b64a074ba7d6 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:16:53.154197 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0d42c1-a1fd-4fd7-a964-73bca16dd72a None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:16:53.239985 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0dbc27-e7fb-42ce-afaa-93553ec6394c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:16:53.240590 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0dbc27-e7fb-42ce-afaa-93553ec6394c None admin] REQUEST_METHOD: `PUT` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:16:53.240873 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0dbc27-e7fb-42ce-afaa-93553ec6394c None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:16:53.241120 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0dbc27-e7fb-42ce-afaa-93553ec6394c None admin] PATH_INFO: `/v3/projects/b57372dfc633444d9f127657ca57d59c/users/2ac071cb56f741a6ab677be4e4baa126/roles/ea4c814713b149108e488de855b025e4` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:16:53.243138 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0dbc27-e7fb-42ce-afaa-93553ec6394c None admin] RBAC: Authorizing `identity:create_grant(project_id=b57372dfc633444d9f127657ca57d59c, user_id=2ac071cb56f741a6ab677be4e4baa126, role_id=ea4c814713b149108e488de855b025e4)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:16:53.244591 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0dbc27-e7fb-42ce-afaa-93553ec6394c None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:16:53.278679 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16648aac-b75c-4f14-bbca-b64a074ba7d6 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:16:53.314751 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a22677-b734-4c1e-88fe-6e3add3b324e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:16:53.315700 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a22677-b734-4c1e-88fe-6e3add3b324e None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:16:53.316158 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a22677-b734-4c1e-88fe-6e3add3b324e None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:16:53.316610 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a22677-b734-4c1e-88fe-6e3add3b324e None admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:16:53.317485 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a22677-b734-4c1e-88fe-6e3add3b324e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:16:53.318580 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a22677-b734-4c1e-88fe-6e3add3b324e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:16:53.320276 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a22677-b734-4c1e-88fe-6e3add3b324e None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:16:53.330002 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 108/230] 10.222.0.30 () {56 vars in 1111 bytes} [Tue Sep 6 04:16:53 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:16:53.342339 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a22677-b734-4c1e-88fe-6e3add3b324e None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:16:53.360168 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 123/231] 10.222.0.30 () {58 vars in 1439 bytes} [Tue Sep 6 04:16:52 2022] PUT /identity/v3/projects/b57372dfc633444d9f127657ca57d59c/users/2ac071cb56f741a6ab677be4e4baa126/roles/ea4c814713b149108e488de855b025e4 => generated 0 bytes in 387 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 04:16:53.384703 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:16:53.477822 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a22677-b734-4c1e-88fe-6e3add3b324e None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:16:53.533008 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ccd526-7311-4140-a421-5342007bccda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:16:53.533617 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ccd526-7311-4140-a421-5342007bccda None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:16:53.533896 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ccd526-7311-4140-a421-5342007bccda None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:16:53.534166 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ccd526-7311-4140-a421-5342007bccda None admin] PATH_INFO: `/v3/users/2ac071cb56f741a6ab677be4e4baa126` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:16:53.537220 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ccd526-7311-4140-a421-5342007bccda None admin] RBAC: Authorizing `identity:get_user(user_id=2ac071cb56f741a6ab677be4e4baa126)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:16:53.538726 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ccd526-7311-4140-a421-5342007bccda None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:16:53.540631 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 109/232] 10.222.0.30 () {56 vars in 1177 bytes} [Tue Sep 6 04:16:53 2022] GET /identity/v3/users/2ac071cb56f741a6ab677be4e4baa126 => generated 283 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:16:53.546430 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0dbc27-e7fb-42ce-afaa-93553ec6394c None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:16:53.658189 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:16:53.674777 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0dbc27-e7fb-42ce-afaa-93553ec6394c None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:16:53.714721 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e66b6b8-df5d-4f14-843e-0a6fbf624af5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:16:53.715375 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e66b6b8-df5d-4f14-843e-0a6fbf624af5 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:16:53.715656 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e66b6b8-df5d-4f14-843e-0a6fbf624af5 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:16:53.715933 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e66b6b8-df5d-4f14-843e-0a6fbf624af5 None admin] PATH_INFO: `/v3/projects/b57372dfc633444d9f127657ca57d59c` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:16:53.717265 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e66b6b8-df5d-4f14-843e-0a6fbf624af5 None admin] RBAC: Authorizing `identity:get_project(project_id=b57372dfc633444d9f127657ca57d59c)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:16:53.719308 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e66b6b8-df5d-4f14-843e-0a6fbf624af5 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:16:53.721981 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 124/233] 10.222.0.30 () {56 vars in 1186 bytes} [Tue Sep 6 04:16:53 2022] GET /identity/v3/projects/b57372dfc633444d9f127657ca57d59c => generated 302 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:16:53.728341 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ccd526-7311-4140-a421-5342007bccda None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:16:53.889060 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ccd526-7311-4140-a421-5342007bccda None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:16:53.890733 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:16:53.932392 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d352e4a-aed3-43d9-aaa6-7c1db691d46d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:16:53.934547 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d352e4a-aed3-43d9-aaa6-7c1db691d46d None admin] REQUEST_METHOD: `PUT` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:16:53.935038 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d352e4a-aed3-43d9-aaa6-7c1db691d46d None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:16:53.935530 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d352e4a-aed3-43d9-aaa6-7c1db691d46d None admin] PATH_INFO: `/v3/projects/b57372dfc633444d9f127657ca57d59c/users/2ac071cb56f741a6ab677be4e4baa126/roles/77197446dd2541b9a4fab9a654c08fc9` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:16:53.937942 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d352e4a-aed3-43d9-aaa6-7c1db691d46d None admin] RBAC: Authorizing `identity:create_grant(project_id=b57372dfc633444d9f127657ca57d59c, user_id=2ac071cb56f741a6ab677be4e4baa126, role_id=77197446dd2541b9a4fab9a654c08fc9)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:16:53.940285 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d352e4a-aed3-43d9-aaa6-7c1db691d46d None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:16:54.044414 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 110/234] 10.222.0.30 () {58 vars in 1439 bytes} [Tue Sep 6 04:16:53 2022] PUT /identity/v3/projects/b57372dfc633444d9f127657ca57d59c/users/2ac071cb56f741a6ab677be4e4baa126/roles/77197446dd2541b9a4fab9a654c08fc9 => generated 0 bytes in 317 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 04:16:54.163679 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a647b36-ccac-445a-ad4d-dc9b431a6099 None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:16:54.164215 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a647b36-ccac-445a-ad4d-dc9b431a6099 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:16:54.164215 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a647b36-ccac-445a-ad4d-dc9b431a6099 None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:16:54.165989 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 125/235] 10.222.0.30 () {54 vars in 927 bytes} [Tue Sep 6 04:16:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 04:16:54.171854 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e672115b-701d-408f-ac22-5c06f7880399 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:16:54.172177 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e672115b-701d-408f-ac22-5c06f7880399 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:16:54.172417 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e672115b-701d-408f-ac22-5c06f7880399 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:16:54.203819 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-e672115b-701d-408f-ac22-5c06f7880399 None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:16:54.205059 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-e672115b-701d-408f-ac22-5c06f7880399 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:16:54.293549 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 111/236] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Sep 6 04:16:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 04:16:54.353819 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc1322d-430e-4982-9152-8e883ea8d0ee None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:16:54.354433 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc1322d-430e-4982-9152-8e883ea8d0ee None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:16:54.354934 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc1322d-430e-4982-9152-8e883ea8d0ee None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:16:54.356766 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 126/237] 10.222.0.30 () {54 vars in 927 bytes} [Tue Sep 6 04:16:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 04:16:54.361883 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6c5bb0-e46b-4cbf-8913-bd14407fbe7d None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:16:54.362169 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6c5bb0-e46b-4cbf-8913-bd14407fbe7d None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:16:54.362444 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6c5bb0-e46b-4cbf-8913-bd14407fbe7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:16:54.383298 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-ec6c5bb0-e46b-4cbf-8913-bd14407fbe7d None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:16:54.384615 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-ec6c5bb0-e46b-4cbf-8913-bd14407fbe7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:16:54.464002 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 112/238] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Sep 6 04:16:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 04:16:54.735808 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc1322d-430e-4982-9152-8e883ea8d0ee None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:16:54.859641 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc1322d-430e-4982-9152-8e883ea8d0ee None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:16:54.885469 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6c5bb0-e46b-4cbf-8913-bd14407fbe7d None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:16:54.910901 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f991846-bb42-4eb8-b338-4d51916ec73f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:16:54.911566 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f991846-bb42-4eb8-b338-4d51916ec73f None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:16:54.911863 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f991846-bb42-4eb8-b338-4d51916ec73f None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:16:54.912147 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f991846-bb42-4eb8-b338-4d51916ec73f None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:16:54.912790 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:16:54.934236 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f991846-bb42-4eb8-b338-4d51916ec73f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 04:16:54.940800 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-0f991846-bb42-4eb8-b338-4d51916ec73f None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 06 04:16:54.940800 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 04:16:54.940800 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 04:16:54.940800 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 04:16:54.940800 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 04:16:54.940800 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 04:16:54.940800 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 04:16:54.940800 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 04:16:54.940800 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 04:16:54.940800 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 04:16:54.940800 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 04:16:54.940800 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 04:16:54.940800 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 04:16:54.940800 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 04:16:54.940800 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 04:16:54.940800 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 04:16:54.941896 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 04:16:54.941896 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 04:16:54.941896 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 04:16:54.941896 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 04:16:54.941896 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 04:16:54.941896 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 04:16:54.941896 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 04:16:54.941896 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 04:16:54.941896 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 04:16:54.941896 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 04:16:54.941896 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 04:16:54.941896 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 04:16:54.941896 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 04:16:54.941896 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 04:16:54.941896 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 04:16:54.941896 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 04:16:54.943435 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 04:16:54.943435 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 04:16:54.943435 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 04:16:54.943435 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 04:16:54.943435 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 04:16:54.943435 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 04:16:54.943435 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 04:16:54.943435 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 04:16:54.943435 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 04:16:54.943435 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 04:16:54.943435 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 06 04:16:54.943435 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 04:16:54.943435 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 127/239] 10.222.0.30 () {56 vars in 1099 bytes} [Tue Sep 6 04:16:54 2022] GET /identity/v3/roles/reader => generated 84 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 04:16:54.949379 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f991846-bb42-4eb8-b338-4d51916ec73f None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:16:54.976693 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:16:55.051586 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6c5bb0-e46b-4cbf-8913-bd14407fbe7d None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:16:55.098400 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6dd20a6-a761-4ede-99e5-6ff304ea630e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:16:55.099571 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6dd20a6-a761-4ede-99e5-6ff304ea630e None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:16:55.099998 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6dd20a6-a761-4ede-99e5-6ff304ea630e None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:16:55.100277 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6dd20a6-a761-4ede-99e5-6ff304ea630e None admin] PATH_INFO: `/v3/domains/default` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:16:55.103158 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6dd20a6-a761-4ede-99e5-6ff304ea630e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:16:55.104633 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6dd20a6-a761-4ede-99e5-6ff304ea630e None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:16:55.106255 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 113/240] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Sep 6 04:16:54 2022] GET /identity/v3/domains/default => generated 199 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 04:16:55.113703 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6dd20a6-a761-4ede-99e5-6ff304ea630e None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:16:55.151347 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f991846-bb42-4eb8-b338-4d51916ec73f None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:16:55.214992 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d04216-fe4d-46d6-b70e-7daf97e8e9bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:16:55.216770 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d04216-fe4d-46d6-b70e-7daf97e8e9bb None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:16:55.217541 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d04216-fe4d-46d6-b70e-7daf97e8e9bb None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:16:55.218013 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d04216-fe4d-46d6-b70e-7daf97e8e9bb None admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:16:55.219020 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d04216-fe4d-46d6-b70e-7daf97e8e9bb None admin] RBAC: Adding query filter params (name=reader) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:16:55.219776 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d04216-fe4d-46d6-b70e-7daf97e8e9bb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:16:55.221894 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d04216-fe4d-46d6-b70e-7daf97e8e9bb None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:16:55.225795 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:16:55.229988 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6dd20a6-a761-4ede-99e5-6ff304ea630e None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:16:55.234717 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 128/241] 10.222.0.30 () {56 vars in 1101 bytes} [Tue Sep 6 04:16:54 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:16:55.236148 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fd0f1e-b18d-4343-826f-f0e89ef7538a None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:16:55.236604 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fd0f1e-b18d-4343-826f-f0e89ef7538a None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:16:55.237028 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fd0f1e-b18d-4343-826f-f0e89ef7538a None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:16:55.238580 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 129/242] 10.222.0.30 () {54 vars in 927 bytes} [Tue Sep 6 04:16:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 04:16:55.244637 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2fd0f1e-b18d-4343-826f-f0e89ef7538a None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:16:55.273351 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a39e3f-bd6d-4687-b466-a1484a49c291 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:16:55.274125 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a39e3f-bd6d-4687-b466-a1484a49c291 None admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:16:55.274499 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a39e3f-bd6d-4687-b466-a1484a49c291 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:16:55.274843 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a39e3f-bd6d-4687-b466-a1484a49c291 None admin] PATH_INFO: `/v3/users` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:16:55.275583 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a39e3f-bd6d-4687-b466-a1484a49c291 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:16:55.277339 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a39e3f-bd6d-4687-b466-a1484a49c291 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:16:55.335669 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:16:55.352879 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 114/243] 10.222.0.30 () {60 vars in 1132 bytes} [Tue Sep 6 04:16:55 2022] POST /identity/v3/users => generated 293 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 04:16:55.358890 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0aa485-2261-443f-9c8c-c9906cea7401 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:16:55.359460 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0aa485-2261-443f-9c8c-c9906cea7401 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:16:55.359891 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0aa485-2261-443f-9c8c-c9906cea7401 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:16:55.376385 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2fd0f1e-b18d-4343-826f-f0e89ef7538a None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:16:55.404342 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-7a0aa485-2261-443f-9c8c-c9906cea7401 None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:16:55.405880 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-7a0aa485-2261-443f-9c8c-c9906cea7401 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:16:55.456947 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8344042b-aab4-4961-87fc-5bb6de243586 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:16:55.459851 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8344042b-aab4-4961-87fc-5bb6de243586 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:16:55.460132 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8344042b-aab4-4961-87fc-5bb6de243586 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:16:55.460416 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8344042b-aab4-4961-87fc-5bb6de243586 None admin] PATH_INFO: `/v3/users/472fdc6775594fa3a9c9a96577625ace` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:16:55.492343 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8344042b-aab4-4961-87fc-5bb6de243586 None admin] RBAC: Authorizing `identity:get_user(user_id=472fdc6775594fa3a9c9a96577625ace)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:16:55.495370 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8344042b-aab4-4961-87fc-5bb6de243586 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:16:55.497131 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 130/244] 10.222.0.30 () {56 vars in 1177 bytes} [Tue Sep 6 04:16:55 2022] GET /identity/v3/users/472fdc6775594fa3a9c9a96577625ace => generated 297 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:16:55.503555 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8344042b-aab4-4961-87fc-5bb6de243586 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:16:55.541835 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 115/245] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Sep 6 04:16:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 04:16:55.631247 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8344042b-aab4-4961-87fc-5bb6de243586 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:16:55.668414 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40f3255-cc8b-45e8-8a94-39340d0efb5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:16:55.669037 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40f3255-cc8b-45e8-8a94-39340d0efb5a None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:16:55.669334 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40f3255-cc8b-45e8-8a94-39340d0efb5a None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:16:55.669609 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40f3255-cc8b-45e8-8a94-39340d0efb5a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:16:55.672263 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:16:55.697272 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a99730-f766-4c6a-b89f-e7a352e6faec None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:16:55.697743 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a99730-f766-4c6a-b89f-e7a352e6faec None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:16:55.698186 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a99730-f766-4c6a-b89f-e7a352e6faec None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:16:55.702219 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 116/246] 10.222.0.30 () {54 vars in 927 bytes} [Tue Sep 6 04:16:55 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 04:16:55.707970 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8124ec2d-ba6e-45a5-9029-08e819f356b6 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:16:55.709089 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8124ec2d-ba6e-45a5-9029-08e819f356b6 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:16:55.709678 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8124ec2d-ba6e-45a5-9029-08e819f356b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:16:55.733066 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-8124ec2d-ba6e-45a5-9029-08e819f356b6 None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:16:55.734675 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-8124ec2d-ba6e-45a5-9029-08e819f356b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:16:55.745403 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40f3255-cc8b-45e8-8a94-39340d0efb5a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:16:55.746906 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40f3255-cc8b-45e8-8a94-39340d0efb5a None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:16:55.810860 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 117/247] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Sep 6 04:16:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 06 04:16:55.861297 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 131/248] 10.222.0.30 () {58 vars in 1282 bytes} [Tue Sep 6 04:16:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 06 04:16:55.862810 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8124ec2d-ba6e-45a5-9029-08e819f356b6 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:16:55.869377 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40f3255-cc8b-45e8-8a94-39340d0efb5a None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:16:55.872822 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:16:55.916398 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:16:55.976927 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8124ec2d-ba6e-45a5-9029-08e819f356b6 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:16:55.985666 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40f3255-cc8b-45e8-8a94-39340d0efb5a None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:16:56.015720 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248211ee-9059-420a-9172-8d38a9a83c0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:16:56.016368 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248211ee-9059-420a-9172-8d38a9a83c0e None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:16:56.016742 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248211ee-9059-420a-9172-8d38a9a83c0e None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:16:56.017019 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248211ee-9059-420a-9172-8d38a9a83c0e None admin] PATH_INFO: `/v3/projects/b57372dfc633444d9f127657ca57d59c` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:16:56.018273 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248211ee-9059-420a-9172-8d38a9a83c0e None admin] RBAC: Authorizing `identity:get_project(project_id=b57372dfc633444d9f127657ca57d59c)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:16:56.020004 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248211ee-9059-420a-9172-8d38a9a83c0e None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:16:56.022474 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 118/249] 10.222.0.30 () {56 vars in 1186 bytes} [Tue Sep 6 04:16:55 2022] GET /identity/v3/projects/b57372dfc633444d9f127657ca57d59c => generated 302 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:16:56.025856 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98768f0e-e1a4-4107-be43-eee40e5b23b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:16:56.026523 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98768f0e-e1a4-4107-be43-eee40e5b23b3 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:16:56.026806 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98768f0e-e1a4-4107-be43-eee40e5b23b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:16:56.027086 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98768f0e-e1a4-4107-be43-eee40e5b23b3 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:16:56.031676 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248211ee-9059-420a-9172-8d38a9a83c0e None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:16:56.042177 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98768f0e-e1a4-4107-be43-eee40e5b23b3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 04:16:56.047119 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-98768f0e-e1a4-4107-be43-eee40e5b23b3 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 06 04:16:56.047119 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 04:16:56.047119 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 04:16:56.047119 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 04:16:56.047119 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 04:16:56.047119 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 04:16:56.047119 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 04:16:56.047119 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 04:16:56.047119 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 04:16:56.047119 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 04:16:56.047119 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 04:16:56.047119 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 04:16:56.047119 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 04:16:56.047119 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 04:16:56.047119 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 04:16:56.047119 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 04:16:56.048015 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 04:16:56.048015 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 04:16:56.048015 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 04:16:56.048015 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 04:16:56.048015 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 04:16:56.048015 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 04:16:56.048015 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 04:16:56.048015 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 04:16:56.048015 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 04:16:56.048015 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 04:16:56.048015 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 04:16:56.048015 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 04:16:56.048015 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 04:16:56.048015 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 04:16:56.048015 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 04:16:56.048015 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 04:16:56.048799 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 04:16:56.048799 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 04:16:56.048799 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 04:16:56.048799 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 04:16:56.048799 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 04:16:56.048799 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 04:16:56.048799 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 04:16:56.048799 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 04:16:56.048799 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 04:16:56.048799 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 04:16:56.048799 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 06 04:16:56.048799 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 04:16:56.048799 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 132/250] 10.222.0.30 () {56 vars in 1099 bytes} [Tue Sep 6 04:16:55 2022] GET /identity/v3/roles/member => generated 84 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 04:16:56.057298 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98768f0e-e1a4-4107-be43-eee40e5b23b3 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:16:56.141096 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:16:56.175944 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248211ee-9059-420a-9172-8d38a9a83c0e None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:16:56.191684 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:16:56.203369 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98768f0e-e1a4-4107-be43-eee40e5b23b3 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:16:56.219485 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45aab15d-62cf-4303-a44d-7994a6cf5efa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:16:56.220383 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45aab15d-62cf-4303-a44d-7994a6cf5efa None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:16:56.220660 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45aab15d-62cf-4303-a44d-7994a6cf5efa None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:16:56.221086 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45aab15d-62cf-4303-a44d-7994a6cf5efa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:16:56.222005 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45aab15d-62cf-4303-a44d-7994a6cf5efa None admin] RBAC: Adding query filter params (role.id=3c011524d1df425b902b8b32c1b485c3, scope.project.id=b57372dfc633444d9f127657ca57d59c, user.id=472fdc6775594fa3a9c9a96577625ace) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:16:56.222476 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45aab15d-62cf-4303-a44d-7994a6cf5efa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:16:56.223929 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45aab15d-62cf-4303-a44d-7994a6cf5efa None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:16:56.233962 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 119/251] 10.222.0.30 () {56 vars in 1374 bytes} [Tue Sep 6 04:16:56 2022] GET /identity/v3/role_assignments?user.id=472fdc6775594fa3a9c9a96577625ace&scope.project.id=b57372dfc633444d9f127657ca57d59c&role.id=3c011524d1df425b902b8b32c1b485c3 => generated 259 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:16:56.234193 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45aab15d-62cf-4303-a44d-7994a6cf5efa None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:16:56.247219 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2145f89-35d1-4199-9f84-53af93ee6131 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:16:56.248419 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2145f89-35d1-4199-9f84-53af93ee6131 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:16:56.248696 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2145f89-35d1-4199-9f84-53af93ee6131 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:16:56.248960 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2145f89-35d1-4199-9f84-53af93ee6131 None admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:16:56.249788 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2145f89-35d1-4199-9f84-53af93ee6131 None admin] RBAC: Adding query filter params (name=member) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:16:56.249969 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2145f89-35d1-4199-9f84-53af93ee6131 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:16:56.251369 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2145f89-35d1-4199-9f84-53af93ee6131 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:16:56.263586 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 133/252] 10.222.0.30 () {56 vars in 1101 bytes} [Tue Sep 6 04:16:56 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:16:56.274454 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2145f89-35d1-4199-9f84-53af93ee6131 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:16:56.434482 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45aab15d-62cf-4303-a44d-7994a6cf5efa None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:16:56.476628 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e2f5e4-8547-4f6c-9d69-27ca6030fc55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:16:56.477460 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e2f5e4-8547-4f6c-9d69-27ca6030fc55 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:16:56.477749 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e2f5e4-8547-4f6c-9d69-27ca6030fc55 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:16:56.478028 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e2f5e4-8547-4f6c-9d69-27ca6030fc55 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:16:56.487863 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2145f89-35d1-4199-9f84-53af93ee6131 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:16:56.488160 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:16:56.494709 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e2f5e4-8547-4f6c-9d69-27ca6030fc55 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 04:16:56.499639 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-b4e2f5e4-8547-4f6c-9d69-27ca6030fc55 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 04:16:56.499639 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 04:16:56.499639 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 04:16:56.499639 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 04:16:56.499639 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 04:16:56.499639 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 04:16:56.499639 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 04:16:56.499639 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 04:16:56.499639 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 04:16:56.499639 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 04:16:56.499639 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 04:16:56.499639 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 04:16:56.499639 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 04:16:56.499639 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 04:16:56.499639 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 04:16:56.499639 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 04:16:56.501807 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 04:16:56.501807 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 04:16:56.501807 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 04:16:56.501807 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 04:16:56.501807 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 04:16:56.501807 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 04:16:56.501807 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 04:16:56.501807 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 04:16:56.501807 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 04:16:56.501807 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 04:16:56.501807 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 04:16:56.501807 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 04:16:56.501807 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 04:16:56.501807 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 04:16:56.501807 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 04:16:56.501807 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 04:16:56.502899 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 04:16:56.502899 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 04:16:56.502899 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 04:16:56.502899 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 04:16:56.502899 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 04:16:56.502899 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 04:16:56.502899 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 04:16:56.502899 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 04:16:56.502899 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 04:16:56.502899 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 04:16:56.502899 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 04:16:56.502899 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 04:16:56.502899 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 120/253] 10.222.0.30 () {56 vars in 1096 bytes} [Tue Sep 6 04:16:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 04:16:56.502899 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba7b93c-4192-4bf0-a0cc-99cc0dcddf7d None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:16:56.503611 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba7b93c-4192-4bf0-a0cc-99cc0dcddf7d None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:16:56.503611 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba7b93c-4192-4bf0-a0cc-99cc0dcddf7d None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:16:56.504425 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 121/254] 10.222.0.30 () {54 vars in 927 bytes} [Tue Sep 6 04:16:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 04:16:56.510593 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba7b93c-4192-4bf0-a0cc-99cc0dcddf7d None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:16:56.589826 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91748b64-c033-4c74-b155-b71e121455f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:16:56.590941 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91748b64-c033-4c74-b155-b71e121455f2 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:16:56.591459 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91748b64-c033-4c74-b155-b71e121455f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:16:56.591935 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91748b64-c033-4c74-b155-b71e121455f2 None admin] PATH_INFO: `/v3/users/35ab231490514d90a4c3fcb3d63e63bd` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:16:56.602215 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91748b64-c033-4c74-b155-b71e121455f2 None admin] RBAC: Authorizing `identity:get_user(user_id=35ab231490514d90a4c3fcb3d63e63bd)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:16:56.604008 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91748b64-c033-4c74-b155-b71e121455f2 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:16:56.606249 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 134/255] 10.222.0.30 () {56 vars in 1177 bytes} [Tue Sep 6 04:16:56 2022] GET /identity/v3/users/35ab231490514d90a4c3fcb3d63e63bd => generated 297 bytes in 332 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:16:56.607695 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fffa4c5-c662-4b67-b93f-d227ee242a63 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:16:56.608182 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fffa4c5-c662-4b67-b93f-d227ee242a63 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:16:56.608664 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fffa4c5-c662-4b67-b93f-d227ee242a63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:16:56.629740 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba7b93c-4192-4bf0-a0cc-99cc0dcddf7d None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:16:56.646169 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-1fffa4c5-c662-4b67-b93f-d227ee242a63 None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:16:56.647791 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-1fffa4c5-c662-4b67-b93f-d227ee242a63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:16:56.684941 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:16:56.702691 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b0ab18-4b35-49b9-a992-f31cffd3138f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:16:56.703503 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b0ab18-4b35-49b9-a992-f31cffd3138f None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:16:56.703784 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b0ab18-4b35-49b9-a992-f31cffd3138f None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:16:56.704059 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b0ab18-4b35-49b9-a992-f31cffd3138f None admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:16:56.704893 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b0ab18-4b35-49b9-a992-f31cffd3138f None admin] RBAC: Adding query filter params (name=admin) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:16:56.705068 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b0ab18-4b35-49b9-a992-f31cffd3138f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:16:56.706454 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b0ab18-4b35-49b9-a992-f31cffd3138f None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:16:56.728001 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 122/256] 10.222.0.30 () {56 vars in 1099 bytes} [Tue Sep 6 04:16:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:16:56.728794 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b0ab18-4b35-49b9-a992-f31cffd3138f None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:16:56.748526 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 135/257] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Sep 6 04:16:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 04:16:56.749836 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fffa4c5-c662-4b67-b93f-d227ee242a63 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:16:56.857700 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:16:56.918623 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fffa4c5-c662-4b67-b93f-d227ee242a63 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:16:56.934481 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b0ab18-4b35-49b9-a992-f31cffd3138f None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:16:56.978783 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328d21f4-d789-43f2-8eb3-24df6b3db158 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:16:56.979352 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328d21f4-d789-43f2-8eb3-24df6b3db158 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:16:56.979627 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328d21f4-d789-43f2-8eb3-24df6b3db158 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:16:56.979898 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328d21f4-d789-43f2-8eb3-24df6b3db158 None admin] PATH_INFO: `/v3/users/2ac071cb56f741a6ab677be4e4baa126` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:16:56.981332 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-328d21f4-d789-43f2-8eb3-24df6b3db158 None admin] RBAC: Authorizing `identity:get_user(user_id=2ac071cb56f741a6ab677be4e4baa126)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:16:56.982780 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-328d21f4-d789-43f2-8eb3-24df6b3db158 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:16:56.984532 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 136/258] 10.222.0.30 () {56 vars in 1177 bytes} [Tue Sep 6 04:16:56 2022] GET /identity/v3/users/2ac071cb56f741a6ab677be4e4baa126 => generated 283 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:16:56.987248 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216b65d6-5a1f-4784-b371-28a3b67f406d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:16:56.987886 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216b65d6-5a1f-4784-b371-28a3b67f406d None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:16:56.988214 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216b65d6-5a1f-4784-b371-28a3b67f406d None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:16:56.988491 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216b65d6-5a1f-4784-b371-28a3b67f406d None admin] PATH_INFO: `/v3/projects/b57372dfc633444d9f127657ca57d59c` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:16:56.990472 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328d21f4-d789-43f2-8eb3-24df6b3db158 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:16:56.991441 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216b65d6-5a1f-4784-b371-28a3b67f406d None admin] RBAC: Authorizing `identity:get_project(project_id=b57372dfc633444d9f127657ca57d59c)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:16:56.992877 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216b65d6-5a1f-4784-b371-28a3b67f406d None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:16:56.994642 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 123/259] 10.222.0.30 () {56 vars in 1186 bytes} [Tue Sep 6 04:16:56 2022] GET /identity/v3/projects/b57372dfc633444d9f127657ca57d59c => generated 302 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:16:57.015601 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216b65d6-5a1f-4784-b371-28a3b67f406d None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:16:57.040563 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:16:57.137378 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328d21f4-d789-43f2-8eb3-24df6b3db158 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:16:57.174637 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb87bd9-9f44-4248-b015-1c88002403be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:16:57.175600 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb87bd9-9f44-4248-b015-1c88002403be None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:16:57.176269 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb87bd9-9f44-4248-b015-1c88002403be None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:16:57.177909 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb87bd9-9f44-4248-b015-1c88002403be None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:16:57.193569 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216b65d6-5a1f-4784-b371-28a3b67f406d None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:16:57.235071 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:16:57.253794 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50073ede-3723-4ee3-9cab-04d00baabb7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:16:57.254476 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50073ede-3723-4ee3-9cab-04d00baabb7e None admin] REQUEST_METHOD: `PUT` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:16:57.254754 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50073ede-3723-4ee3-9cab-04d00baabb7e None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:16:57.255739 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50073ede-3723-4ee3-9cab-04d00baabb7e None admin] PATH_INFO: `/v3/projects/b57372dfc633444d9f127657ca57d59c/users/35ab231490514d90a4c3fcb3d63e63bd/roles/29607b2de39f40d68bb6ceef64b8a732` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:16:57.262765 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50073ede-3723-4ee3-9cab-04d00baabb7e None admin] RBAC: Authorizing `identity:create_grant(project_id=b57372dfc633444d9f127657ca57d59c, user_id=35ab231490514d90a4c3fcb3d63e63bd, role_id=29607b2de39f40d68bb6ceef64b8a732)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:16:57.265558 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50073ede-3723-4ee3-9cab-04d00baabb7e None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:16:57.267370 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb87bd9-9f44-4248-b015-1c88002403be None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:16:57.269406 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb87bd9-9f44-4248-b015-1c88002403be None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:16:57.294830 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:16:57.336693 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 124/260] 10.222.0.30 () {58 vars in 1439 bytes} [Tue Sep 6 04:16:57 2022] PUT /identity/v3/projects/b57372dfc633444d9f127657ca57d59c/users/35ab231490514d90a4c3fcb3d63e63bd/roles/29607b2de39f40d68bb6ceef64b8a732 => generated 0 bytes in 321 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 04:16:57.337926 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50073ede-3723-4ee3-9cab-04d00baabb7e None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:16:57.397596 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 137/261] 10.222.0.30 () {58 vars in 1282 bytes} [Tue Sep 6 04:16:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 408 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 06 04:16:57.404121 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb87bd9-9f44-4248-b015-1c88002403be None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:16:57.524107 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50073ede-3723-4ee3-9cab-04d00baabb7e None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:16:57.525344 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb87bd9-9f44-4248-b015-1c88002403be None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:16:57.549909 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:16:57.562812 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53003d2-6fac-4e0a-b188-49e8184ad8f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:16:57.563883 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53003d2-6fac-4e0a-b188-49e8184ad8f9 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:16:57.564356 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53003d2-6fac-4e0a-b188-49e8184ad8f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:16:57.564847 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53003d2-6fac-4e0a-b188-49e8184ad8f9 None admin] PATH_INFO: `/v3/projects/b57372dfc633444d9f127657ca57d59c` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:16:57.566211 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53003d2-6fac-4e0a-b188-49e8184ad8f9 None admin] RBAC: Authorizing `identity:get_project(project_id=b57372dfc633444d9f127657ca57d59c)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:16:57.567934 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53003d2-6fac-4e0a-b188-49e8184ad8f9 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:16:57.570244 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 138/262] 10.222.0.30 () {56 vars in 1186 bytes} [Tue Sep 6 04:16:57 2022] GET /identity/v3/projects/b57372dfc633444d9f127657ca57d59c => generated 302 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:16:57.578492 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53003d2-6fac-4e0a-b188-49e8184ad8f9 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:16:57.622220 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0afaaf88-9c3f-44bd-ab57-fbe2baa238f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:16:57.623171 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afaaf88-9c3f-44bd-ab57-fbe2baa238f6 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:16:57.623632 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afaaf88-9c3f-44bd-ab57-fbe2baa238f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:16:57.624038 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afaaf88-9c3f-44bd-ab57-fbe2baa238f6 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:16:57.654927 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0afaaf88-9c3f-44bd-ab57-fbe2baa238f6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 04:16:57.660140 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-0afaaf88-9c3f-44bd-ab57-fbe2baa238f6 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 06 04:16:57.660140 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 04:16:57.660140 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 04:16:57.660140 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 04:16:57.660140 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 04:16:57.660140 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 04:16:57.660140 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 04:16:57.660140 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 04:16:57.660140 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 04:16:57.660140 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 04:16:57.660140 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 04:16:57.660140 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 04:16:57.660140 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 04:16:57.660140 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 04:16:57.660140 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 04:16:57.660140 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 04:16:57.661029 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 04:16:57.661029 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 04:16:57.661029 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 04:16:57.661029 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 04:16:57.661029 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 04:16:57.661029 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 04:16:57.661029 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 04:16:57.661029 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 04:16:57.661029 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 04:16:57.661029 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 04:16:57.661029 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 04:16:57.661029 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 04:16:57.661029 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 04:16:57.661029 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 04:16:57.661029 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 04:16:57.661029 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 04:16:57.662426 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 04:16:57.662426 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 04:16:57.662426 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 04:16:57.662426 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 04:16:57.662426 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 04:16:57.662426 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 04:16:57.662426 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 04:16:57.662426 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 04:16:57.662426 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 04:16:57.662426 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 04:16:57.662426 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 06 04:16:57.662426 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 04:16:57.664842 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 125/263] 10.222.0.30 () {56 vars in 1114 bytes} [Tue Sep 6 04:16:57 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 327 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 04:16:57.672370 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0afaaf88-9c3f-44bd-ab57-fbe2baa238f6 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:16:57.732776 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53003d2-6fac-4e0a-b188-49e8184ad8f9 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:16:57.745940 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:16:57.774902 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0a098e-53b3-4b76-b86b-7b578f05953b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:16:57.775770 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0a098e-53b3-4b76-b86b-7b578f05953b None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:16:57.776198 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0a098e-53b3-4b76-b86b-7b578f05953b None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:16:57.776605 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0a098e-53b3-4b76-b86b-7b578f05953b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:16:57.777521 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc0a098e-53b3-4b76-b86b-7b578f05953b None admin] RBAC: Adding query filter params (role.id=ea4c814713b149108e488de855b025e4, scope.project.id=b57372dfc633444d9f127657ca57d59c, user.id=2ac071cb56f741a6ab677be4e4baa126) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:16:57.778025 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc0a098e-53b3-4b76-b86b-7b578f05953b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:16:57.779727 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc0a098e-53b3-4b76-b86b-7b578f05953b None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:16:57.787714 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 139/264] 10.222.0.30 () {56 vars in 1374 bytes} [Tue Sep 6 04:16:57 2022] GET /identity/v3/role_assignments?user.id=2ac071cb56f741a6ab677be4e4baa126&scope.project.id=b57372dfc633444d9f127657ca57d59c&role.id=ea4c814713b149108e488de855b025e4 => generated 608 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:16:57.805861 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0afaaf88-9c3f-44bd-ab57-fbe2baa238f6 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:16:57.851424 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9328f9-e0ca-45f8-bfc9-4569c0f439b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:16:57.852083 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9328f9-e0ca-45f8-bfc9-4569c0f439b2 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:16:57.852369 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9328f9-e0ca-45f8-bfc9-4569c0f439b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:16:57.852646 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9328f9-e0ca-45f8-bfc9-4569c0f439b2 None admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:16:57.853366 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b9328f9-e0ca-45f8-bfc9-4569c0f439b2 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:16:57.853677 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b9328f9-e0ca-45f8-bfc9-4569c0f439b2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:16:57.855136 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b9328f9-e0ca-45f8-bfc9-4569c0f439b2 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:16:57.864641 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 126/265] 10.222.0.30 () {56 vars in 1111 bytes} [Tue Sep 6 04:16:57 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 04:16:57.870791 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0a098e-53b3-4b76-b86b-7b578f05953b None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:16:57.949339 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:16:58.007979 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0a098e-53b3-4b76-b86b-7b578f05953b None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:16:58.049884 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694b5e89-de1e-47eb-a05c-39520713c2d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:16:58.050839 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694b5e89-de1e-47eb-a05c-39520713c2d4 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:16:58.051307 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694b5e89-de1e-47eb-a05c-39520713c2d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:16:58.051747 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694b5e89-de1e-47eb-a05c-39520713c2d4 None admin] PATH_INFO: `/v3/users/2ac071cb56f741a6ab677be4e4baa126` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:16:58.053118 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694b5e89-de1e-47eb-a05c-39520713c2d4 None admin] RBAC: Authorizing `identity:get_user(user_id=2ac071cb56f741a6ab677be4e4baa126)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:16:58.054851 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694b5e89-de1e-47eb-a05c-39520713c2d4 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:16:58.056923 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 140/266] 10.222.0.30 () {56 vars in 1177 bytes} [Tue Sep 6 04:16:57 2022] GET /identity/v3/users/2ac071cb56f741a6ab677be4e4baa126 => generated 283 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:16:58.062413 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9328f9-e0ca-45f8-bfc9-4569c0f439b2 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:16:58.178408 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9328f9-e0ca-45f8-bfc9-4569c0f439b2 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:16:58.189470 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:16:58.213433 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08e8392-d4a5-4e7c-95ee-7f1e8361226d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:16:58.214178 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08e8392-d4a5-4e7c-95ee-7f1e8361226d None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:16:58.214356 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08e8392-d4a5-4e7c-95ee-7f1e8361226d None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:16:58.214625 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08e8392-d4a5-4e7c-95ee-7f1e8361226d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:16:58.287726 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08e8392-d4a5-4e7c-95ee-7f1e8361226d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:16:58.289208 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08e8392-d4a5-4e7c-95ee-7f1e8361226d None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:16:58.393140 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 127/267] 10.222.0.30 () {58 vars in 1282 bytes} [Tue Sep 6 04:16:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 06 04:16:58.399553 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694b5e89-de1e-47eb-a05c-39520713c2d4 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:16:58.537107 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694b5e89-de1e-47eb-a05c-39520713c2d4 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:16:58.573291 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:16:58.578516 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec90829-4078-4989-ab94-4e77e977df1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:16:58.579173 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec90829-4078-4989-ab94-4e77e977df1c None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:16:58.579455 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec90829-4078-4989-ab94-4e77e977df1c None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:16:58.579729 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec90829-4078-4989-ab94-4e77e977df1c None admin] PATH_INFO: `/v3/projects/b57372dfc633444d9f127657ca57d59c` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:16:58.580864 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec90829-4078-4989-ab94-4e77e977df1c None admin] RBAC: Authorizing `identity:get_project(project_id=b57372dfc633444d9f127657ca57d59c)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:16:58.582358 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec90829-4078-4989-ab94-4e77e977df1c None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:16:58.588441 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 141/268] 10.222.0.30 () {56 vars in 1186 bytes} [Tue Sep 6 04:16:58 2022] GET /identity/v3/projects/b57372dfc633444d9f127657ca57d59c => generated 302 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:16:58.591362 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08e8392-d4a5-4e7c-95ee-7f1e8361226d None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:16:58.609434 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:16:58.641060 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ee944c-97bb-4bbc-8735-e8c4d13443eb None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:16:58.641352 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ee944c-97bb-4bbc-8735-e8c4d13443eb None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:16:58.641565 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ee944c-97bb-4bbc-8735-e8c4d13443eb None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:16:58.642956 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 142/269] 10.222.0.30 () {54 vars in 927 bytes} [Tue Sep 6 04:16:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 04:16:58.648993 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57b11ed-0f19-489c-86b4-0fc5ff408ab9 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:16:58.649277 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57b11ed-0f19-489c-86b4-0fc5ff408ab9 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:16:58.649581 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57b11ed-0f19-489c-86b4-0fc5ff408ab9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:16:58.682948 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-b57b11ed-0f19-489c-86b4-0fc5ff408ab9 None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:16:58.684151 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-b57b11ed-0f19-489c-86b4-0fc5ff408ab9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:16:58.756681 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08e8392-d4a5-4e7c-95ee-7f1e8361226d None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:16:58.815578 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 143/270] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Sep 6 04:16:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 06 04:16:58.826062 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35328dd9-3acf-4e11-bfc4-8ebdb861554e None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:16:58.826392 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35328dd9-3acf-4e11-bfc4-8ebdb861554e None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:16:58.826713 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35328dd9-3acf-4e11-bfc4-8ebdb861554e None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:16:58.828406 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 144/271] 10.222.0.30 () {54 vars in 927 bytes} [Tue Sep 6 04:16:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 04:16:58.829157 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a3af38-0851-4290-b4aa-491438c5f5a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:16:58.829832 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a3af38-0851-4290-b4aa-491438c5f5a3 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:16:58.830110 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a3af38-0851-4290-b4aa-491438c5f5a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:16:58.830378 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a3af38-0851-4290-b4aa-491438c5f5a3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:16:58.831127 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a3af38-0851-4290-b4aa-491438c5f5a3 None admin] RBAC: Adding query filter params (role.id=77197446dd2541b9a4fab9a654c08fc9, scope.project.id=b57372dfc633444d9f127657ca57d59c, user.id=2ac071cb56f741a6ab677be4e4baa126) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:16:58.831432 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a3af38-0851-4290-b4aa-491438c5f5a3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:16:58.832892 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a3af38-0851-4290-b4aa-491438c5f5a3 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:16:58.838772 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39661fb2-16ee-4c46-9332-c9c1b1cf3607 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:16:58.839049 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39661fb2-16ee-4c46-9332-c9c1b1cf3607 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:16:58.839506 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39661fb2-16ee-4c46-9332-c9c1b1cf3607 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:16:58.856308 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 128/272] 10.222.0.30 () {56 vars in 1374 bytes} [Tue Sep 6 04:16:58 2022] GET /identity/v3/role_assignments?user.id=2ac071cb56f741a6ab677be4e4baa126&scope.project.id=b57372dfc633444d9f127657ca57d59c&role.id=77197446dd2541b9a4fab9a654c08fc9 => generated 608 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:16:58.873910 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-39661fb2-16ee-4c46-9332-c9c1b1cf3607 None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:16:58.875111 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-39661fb2-16ee-4c46-9332-c9c1b1cf3607 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:16:58.967523 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 145/273] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Sep 6 04:16:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 06 04:16:59.092081 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1286f8a-d796-4d1c-935c-2b71e3f41729 None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:16:59.092367 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1286f8a-d796-4d1c-935c-2b71e3f41729 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:16:59.092645 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1286f8a-d796-4d1c-935c-2b71e3f41729 None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:16:59.094154 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 129/274] 10.222.0.30 () {54 vars in 927 bytes} [Tue Sep 6 04:16:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 04:16:59.099648 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96feec82-f2a2-4d6e-8621-32f83546ce1e None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:16:59.100042 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96feec82-f2a2-4d6e-8621-32f83546ce1e None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:16:59.100320 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96feec82-f2a2-4d6e-8621-32f83546ce1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:16:59.119220 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-96feec82-f2a2-4d6e-8621-32f83546ce1e None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:16:59.120394 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-96feec82-f2a2-4d6e-8621-32f83546ce1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:16:59.163023 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1286f8a-d796-4d1c-935c-2b71e3f41729 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:16:59.176789 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:16:59.205996 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 146/275] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Sep 6 04:16:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 04:16:59.251400 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:16:59.278332 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1286f8a-d796-4d1c-935c-2b71e3f41729 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:16:59.312587 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed81180a-cbe3-48e6-b078-31d3f763e87d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:16:59.313223 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed81180a-cbe3-48e6-b078-31d3f763e87d None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:16:59.313534 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed81180a-cbe3-48e6-b078-31d3f763e87d None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:16:59.313802 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed81180a-cbe3-48e6-b078-31d3f763e87d None admin] PATH_INFO: `/v3/domains/default` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:16:59.314907 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed81180a-cbe3-48e6-b078-31d3f763e87d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:16:59.316538 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96feec82-f2a2-4d6e-8621-32f83546ce1e None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:16:59.318098 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed81180a-cbe3-48e6-b078-31d3f763e87d None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:16:59.319784 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 130/276] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Sep 6 04:16:59 2022] GET /identity/v3/domains/default => generated 199 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 04:16:59.326236 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed81180a-cbe3-48e6-b078-31d3f763e87d None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:16:59.460083 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed81180a-cbe3-48e6-b078-31d3f763e87d None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:16:59.481591 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96feec82-f2a2-4d6e-8621-32f83546ce1e None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:16:59.499114 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f80a3f-917c-493e-9d09-2e72bc34fb39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:16:59.500267 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f80a3f-917c-493e-9d09-2e72bc34fb39 None admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:16:59.500789 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f80a3f-917c-493e-9d09-2e72bc34fb39 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:16:59.501249 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f80a3f-917c-493e-9d09-2e72bc34fb39 None admin] PATH_INFO: `/v3/users` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:16:59.502175 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f80a3f-917c-493e-9d09-2e72bc34fb39 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:16:59.504310 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f80a3f-917c-493e-9d09-2e72bc34fb39 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:16:59.519656 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409c01b3-1af0-403f-942b-878b135bbfb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:16:59.520295 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409c01b3-1af0-403f-942b-878b135bbfb7 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:16:59.520571 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409c01b3-1af0-403f-942b-878b135bbfb7 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:16:59.521444 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409c01b3-1af0-403f-942b-878b135bbfb7 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:16:59.531567 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409c01b3-1af0-403f-942b-878b135bbfb7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 04:16:59.536366 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-409c01b3-1af0-403f-942b-878b135bbfb7 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 06 04:16:59.536366 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 04:16:59.536366 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 04:16:59.536366 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 04:16:59.536366 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 04:16:59.536366 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 04:16:59.536366 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 04:16:59.536366 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 04:16:59.536366 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 04:16:59.536366 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 04:16:59.536366 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 04:16:59.536366 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 04:16:59.536366 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 04:16:59.536366 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 04:16:59.536366 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 04:16:59.536366 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 04:16:59.537297 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 04:16:59.537297 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 04:16:59.537297 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 04:16:59.537297 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 04:16:59.537297 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 04:16:59.537297 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 04:16:59.537297 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 04:16:59.537297 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 04:16:59.537297 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 04:16:59.537297 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 04:16:59.537297 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 04:16:59.537297 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 04:16:59.537297 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 04:16:59.537297 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 04:16:59.537297 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 04:16:59.537297 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 04:16:59.538132 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 04:16:59.538132 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 04:16:59.538132 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 04:16:59.538132 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 04:16:59.538132 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 04:16:59.538132 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 04:16:59.538132 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 04:16:59.538132 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 04:16:59.538132 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 04:16:59.538132 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 04:16:59.538132 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 06 04:16:59.538132 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 04:16:59.538132 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 147/277] 10.222.0.30 () {56 vars in 1099 bytes} [Tue Sep 6 04:16:59 2022] GET /identity/v3/roles/member => generated 84 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 04:16:59.538807 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409c01b3-1af0-403f-942b-878b135bbfb7 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:16:59.579469 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:16:59.608034 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:16:59.613763 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 131/278] 10.222.0.30 () {60 vars in 1132 bytes} [Tue Sep 6 04:16:59 2022] POST /identity/v3/users => generated 293 bytes in 288 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 04:16:59.622414 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f80a3f-917c-493e-9d09-2e72bc34fb39 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:16:59.645552 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409c01b3-1af0-403f-942b-878b135bbfb7 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:16:59.695548 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515f4259-9366-4876-ab0e-e9b211647052 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:16:59.696724 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515f4259-9366-4876-ab0e-e9b211647052 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:16:59.697017 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515f4259-9366-4876-ab0e-e9b211647052 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:16:59.697297 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515f4259-9366-4876-ab0e-e9b211647052 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:16:59.716584 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515f4259-9366-4876-ab0e-e9b211647052 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 04:16:59.721647 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-515f4259-9366-4876-ab0e-e9b211647052 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 06 04:16:59.721647 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 04:16:59.721647 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 04:16:59.721647 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 04:16:59.721647 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 04:16:59.721647 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 04:16:59.721647 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 04:16:59.721647 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 04:16:59.721647 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 04:16:59.721647 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 04:16:59.721647 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 04:16:59.721647 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 04:16:59.721647 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 04:16:59.721647 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 04:16:59.721647 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 04:16:59.721647 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 04:16:59.722587 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 04:16:59.722587 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 04:16:59.722587 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 04:16:59.722587 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 04:16:59.722587 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 04:16:59.722587 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 04:16:59.722587 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 04:16:59.722587 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 04:16:59.722587 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 04:16:59.722587 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 04:16:59.722587 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 04:16:59.722587 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 04:16:59.722587 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 04:16:59.722587 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 04:16:59.722587 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 04:16:59.722587 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 04:16:59.723390 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 04:16:59.723390 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 04:16:59.723390 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 04:16:59.723390 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 04:16:59.723390 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 04:16:59.723390 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 04:16:59.723390 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 04:16:59.723390 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 04:16:59.723390 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 04:16:59.723390 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 04:16:59.723390 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 06 04:16:59.723390 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 04:16:59.723390 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 148/279] 10.222.0.30 () {56 vars in 1099 bytes} [Tue Sep 6 04:16:59 2022] GET /identity/v3/roles/reader => generated 84 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 04:16:59.730362 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515f4259-9366-4876-ab0e-e9b211647052 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:16:59.766649 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f80a3f-917c-493e-9d09-2e72bc34fb39 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:16:59.812232 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a9eace-3260-4329-b2ee-76aec24a8afc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:16:59.812863 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a9eace-3260-4329-b2ee-76aec24a8afc None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:16:59.813144 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a9eace-3260-4329-b2ee-76aec24a8afc None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:16:59.813451 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a9eace-3260-4329-b2ee-76aec24a8afc None admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:16:59.814133 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a9eace-3260-4329-b2ee-76aec24a8afc None admin] RBAC: Adding query filter params (name=member) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:16:59.814438 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a9eace-3260-4329-b2ee-76aec24a8afc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:16:59.815830 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a9eace-3260-4329-b2ee-76aec24a8afc None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:16:59.825242 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 132/280] 10.222.0.30 () {56 vars in 1101 bytes} [Tue Sep 6 04:16:59 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:16:59.832027 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a9eace-3260-4329-b2ee-76aec24a8afc None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:16:59.871920 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:16:59.875215 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515f4259-9366-4876-ab0e-e9b211647052 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:16:59.922424 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:16:59.926900 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70109cbd-c0de-4631-873a-3344cd644dc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:16:59.927567 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70109cbd-c0de-4631-873a-3344cd644dc6 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:16:59.927878 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70109cbd-c0de-4631-873a-3344cd644dc6 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:16:59.928153 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70109cbd-c0de-4631-873a-3344cd644dc6 None admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:16:59.928832 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70109cbd-c0de-4631-873a-3344cd644dc6 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:16:59.929139 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70109cbd-c0de-4631-873a-3344cd644dc6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:16:59.930611 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70109cbd-c0de-4631-873a-3344cd644dc6 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:16:59.941436 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 149/281] 10.222.0.30 () {56 vars in 1101 bytes} [Tue Sep 6 04:16:59 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:16:59.942792 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295471b8-1265-46ea-8604-c05b776807bd None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:16:59.943138 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295471b8-1265-46ea-8604-c05b776807bd None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:16:59.943471 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295471b8-1265-46ea-8604-c05b776807bd None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:16:59.947420 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 150/282] 10.222.0.30 () {54 vars in 927 bytes} [Tue Sep 6 04:16:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 04:16:59.951258 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295471b8-1265-46ea-8604-c05b776807bd None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:16:59.979906 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a9eace-3260-4329-b2ee-76aec24a8afc None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:00.016884 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76cbc120-bbd5-4d63-9fc8-64641ef9d882 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:00.018009 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cbc120-bbd5-4d63-9fc8-64641ef9d882 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:00.018574 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cbc120-bbd5-4d63-9fc8-64641ef9d882 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:00.019061 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cbc120-bbd5-4d63-9fc8-64641ef9d882 None admin] PATH_INFO: `/v3/users/36df010d3b9d4a8e89f1dba7452b7341` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:00.034618 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76cbc120-bbd5-4d63-9fc8-64641ef9d882 None admin] RBAC: Authorizing `identity:get_user(user_id=36df010d3b9d4a8e89f1dba7452b7341)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:17:00.036663 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76cbc120-bbd5-4d63-9fc8-64641ef9d882 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:17:00.038874 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 133/283] 10.222.0.30 () {56 vars in 1177 bytes} [Tue Sep 6 04:16:59 2022] GET /identity/v3/users/36df010d3b9d4a8e89f1dba7452b7341 => generated 293 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 04:17:00.040711 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b250de74-cede-4012-ad8b-f8e8fc3a1369 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:00.041199 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b250de74-cede-4012-ad8b-f8e8fc3a1369 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:00.041695 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b250de74-cede-4012-ad8b-f8e8fc3a1369 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:00.061553 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-b250de74-cede-4012-ad8b-f8e8fc3a1369 None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:17:00.063214 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-b250de74-cede-4012-ad8b-f8e8fc3a1369 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:17:00.074401 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295471b8-1265-46ea-8604-c05b776807bd None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:00.111810 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9adb0cf-5604-4c6f-8cce-ea5fd4c31abb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:00.112458 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9adb0cf-5604-4c6f-8cce-ea5fd4c31abb None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:00.112739 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9adb0cf-5604-4c6f-8cce-ea5fd4c31abb None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:00.113342 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9adb0cf-5604-4c6f-8cce-ea5fd4c31abb None admin] PATH_INFO: `/v3/users/472fdc6775594fa3a9c9a96577625ace` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:00.114381 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9adb0cf-5604-4c6f-8cce-ea5fd4c31abb None admin] RBAC: Authorizing `identity:get_user(user_id=472fdc6775594fa3a9c9a96577625ace)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:17:00.116690 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9adb0cf-5604-4c6f-8cce-ea5fd4c31abb None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:17:00.118891 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 151/284] 10.222.0.30 () {56 vars in 1177 bytes} [Tue Sep 6 04:16:59 2022] GET /identity/v3/users/472fdc6775594fa3a9c9a96577625ace => generated 297 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 04:17:00.119776 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9adb0cf-5604-4c6f-8cce-ea5fd4c31abb None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:00.140214 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 134/285] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Sep 6 04:17:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 04:17:00.147559 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b250de74-cede-4012-ad8b-f8e8fc3a1369 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:00.153291 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:17:00.229634 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:17:00.231872 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9adb0cf-5604-4c6f-8cce-ea5fd4c31abb None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:00.269506 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f9047a-b4a4-4b45-ba3f-da59e5c33591 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:00.269925 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f9047a-b4a4-4b45-ba3f-da59e5c33591 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:00.270213 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f9047a-b4a4-4b45-ba3f-da59e5c33591 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:00.270530 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f9047a-b4a4-4b45-ba3f-da59e5c33591 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:00.272521 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f9047a-b4a4-4b45-ba3f-da59e5c33591 None admin] RBAC: Adding query filter params (role.id=29607b2de39f40d68bb6ceef64b8a732, user.id=36df010d3b9d4a8e89f1dba7452b7341, scope.system=all) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:17:00.272836 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f9047a-b4a4-4b45-ba3f-da59e5c33591 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:17:00.274368 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f9047a-b4a4-4b45-ba3f-da59e5c33591 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:17:00.280562 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 152/286] 10.222.0.30 () {56 vars in 1308 bytes} [Tue Sep 6 04:17:00 2022] GET /identity/v3/role_assignments?user.id=36df010d3b9d4a8e89f1dba7452b7341&scope.system=all&role.id=29607b2de39f40d68bb6ceef64b8a732 => generated 226 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:17:00.288389 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b250de74-cede-4012-ad8b-f8e8fc3a1369 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:00.328929 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2410befe-3555-4f16-9176-bbe6888b3d98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:00.330033 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2410befe-3555-4f16-9176-bbe6888b3d98 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:00.330518 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2410befe-3555-4f16-9176-bbe6888b3d98 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:00.331011 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2410befe-3555-4f16-9176-bbe6888b3d98 None admin] PATH_INFO: `/v3/projects/b57372dfc633444d9f127657ca57d59c` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:00.332583 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2410befe-3555-4f16-9176-bbe6888b3d98 None admin] RBAC: Authorizing `identity:get_project(project_id=b57372dfc633444d9f127657ca57d59c)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:17:00.334422 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2410befe-3555-4f16-9176-bbe6888b3d98 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:17:00.336488 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 135/287] 10.222.0.30 () {56 vars in 1186 bytes} [Tue Sep 6 04:17:00 2022] GET /identity/v3/projects/b57372dfc633444d9f127657ca57d59c => generated 302 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:17:00.342608 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f9047a-b4a4-4b45-ba3f-da59e5c33591 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:00.490804 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:17:00.497706 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f9047a-b4a4-4b45-ba3f-da59e5c33591 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:00.533635 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b756b158-c9e8-4a42-9fb3-1d633ca179b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:00.534227 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b756b158-c9e8-4a42-9fb3-1d633ca179b0 None admin] REQUEST_METHOD: `PUT` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:00.534510 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b756b158-c9e8-4a42-9fb3-1d633ca179b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:00.534788 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b756b158-c9e8-4a42-9fb3-1d633ca179b0 None admin] PATH_INFO: `/v3/projects/b57372dfc633444d9f127657ca57d59c/users/472fdc6775594fa3a9c9a96577625ace/roles/3c011524d1df425b902b8b32c1b485c3` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:00.537244 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b756b158-c9e8-4a42-9fb3-1d633ca179b0 None admin] RBAC: Authorizing `identity:create_grant(project_id=b57372dfc633444d9f127657ca57d59c, user_id=472fdc6775594fa3a9c9a96577625ace, role_id=3c011524d1df425b902b8b32c1b485c3)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:17:00.538789 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b756b158-c9e8-4a42-9fb3-1d633ca179b0 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:17:00.546458 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2410befe-3555-4f16-9176-bbe6888b3d98 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:00.625904 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 153/288] 10.222.0.30 () {58 vars in 1439 bytes} [Tue Sep 6 04:17:00 2022] PUT /identity/v3/projects/b57372dfc633444d9f127657ca57d59c/users/472fdc6775594fa3a9c9a96577625ace/roles/3c011524d1df425b902b8b32c1b485c3 => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 04:17:00.672407 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2410befe-3555-4f16-9176-bbe6888b3d98 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:00.701421 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:17:00.725283 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe09771-670c-4386-b6e0-df80092001d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:00.726305 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe09771-670c-4386-b6e0-df80092001d6 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:00.726767 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe09771-670c-4386-b6e0-df80092001d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:00.727235 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe09771-670c-4386-b6e0-df80092001d6 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:00.743561 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe09771-670c-4386-b6e0-df80092001d6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 04:17:00.749081 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-bbe09771-670c-4386-b6e0-df80092001d6 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 06 04:17:00.749081 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 04:17:00.749081 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 04:17:00.749081 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 04:17:00.749081 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 04:17:00.749081 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 04:17:00.749081 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 04:17:00.749081 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 04:17:00.749081 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 04:17:00.749081 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 04:17:00.749081 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 04:17:00.749081 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 04:17:00.749081 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 04:17:00.749081 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 04:17:00.749081 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 04:17:00.749081 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 04:17:00.750025 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 04:17:00.750025 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 04:17:00.750025 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 04:17:00.750025 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 04:17:00.750025 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 04:17:00.750025 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 04:17:00.750025 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 04:17:00.750025 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 04:17:00.750025 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 04:17:00.750025 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 04:17:00.750025 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 04:17:00.750025 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 04:17:00.750025 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 04:17:00.750025 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 04:17:00.750025 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 04:17:00.750025 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 04:17:00.750833 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 04:17:00.750833 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 04:17:00.750833 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 04:17:00.750833 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 04:17:00.750833 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 04:17:00.750833 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 04:17:00.750833 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 04:17:00.750833 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 04:17:00.750833 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 04:17:00.750833 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 04:17:00.750833 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 06 04:17:00.750833 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 04:17:00.753499 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 136/289] 10.222.0.30 () {56 vars in 1099 bytes} [Tue Sep 6 04:17:00 2022] GET /identity/v3/roles/member => generated 84 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 04:17:00.759703 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b756b158-c9e8-4a42-9fb3-1d633ca179b0 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:00.893783 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b756b158-c9e8-4a42-9fb3-1d633ca179b0 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:00.935835 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72120ee-9c18-4c88-af3a-e2fbbf91b113 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:00.936454 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72120ee-9c18-4c88-af3a-e2fbbf91b113 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:00.936735 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72120ee-9c18-4c88-af3a-e2fbbf91b113 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:00.937002 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72120ee-9c18-4c88-af3a-e2fbbf91b113 None admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:00.937701 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72120ee-9c18-4c88-af3a-e2fbbf91b113 None admin] RBAC: Adding query filter params (name=member) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:17:00.938002 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72120ee-9c18-4c88-af3a-e2fbbf91b113 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:17:00.939388 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72120ee-9c18-4c88-af3a-e2fbbf91b113 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:17:00.948995 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 154/290] 10.222.0.30 () {56 vars in 1101 bytes} [Tue Sep 6 04:17:00 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:17:00.954287 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe09771-670c-4386-b6e0-df80092001d6 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:01.086100 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe09771-670c-4386-b6e0-df80092001d6 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:01.141123 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ee0014-6b7a-4792-ad3e-0017dd337680 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:01.141822 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ee0014-6b7a-4792-ad3e-0017dd337680 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:01.142106 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ee0014-6b7a-4792-ad3e-0017dd337680 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:01.142383 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ee0014-6b7a-4792-ad3e-0017dd337680 None admin] PATH_INFO: `/v3/users/35ab231490514d90a4c3fcb3d63e63bd` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:01.143584 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ee0014-6b7a-4792-ad3e-0017dd337680 None admin] RBAC: Authorizing `identity:get_user(user_id=35ab231490514d90a4c3fcb3d63e63bd)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:17:01.145144 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ee0014-6b7a-4792-ad3e-0017dd337680 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:17:01.147906 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 137/291] 10.222.0.30 () {56 vars in 1177 bytes} [Tue Sep 6 04:17:00 2022] GET /identity/v3/users/35ab231490514d90a4c3fcb3d63e63bd => generated 297 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 04:17:01.153897 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72120ee-9c18-4c88-af3a-e2fbbf91b113 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:01.181945 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:17:01.302183 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72120ee-9c18-4c88-af3a-e2fbbf91b113 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:01.344732 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c5bd89-3977-4c24-98c2-e0938b3bd974 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:01.347735 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c5bd89-3977-4c24-98c2-e0938b3bd974 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:01.348217 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c5bd89-3977-4c24-98c2-e0938b3bd974 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:01.348686 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c5bd89-3977-4c24-98c2-e0938b3bd974 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:01.434858 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c5bd89-3977-4c24-98c2-e0938b3bd974 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:17:01.436619 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c5bd89-3977-4c24-98c2-e0938b3bd974 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:17:01.454858 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:17:01.543708 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 155/292] 10.222.0.30 () {58 vars in 1282 bytes} [Tue Sep 6 04:17:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 06 04:17:01.550370 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ee0014-6b7a-4792-ad3e-0017dd337680 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:01.633176 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:17:01.713083 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ee0014-6b7a-4792-ad3e-0017dd337680 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:01.759536 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6c5882-2e5d-4b55-a028-53ef4e871584 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:01.760194 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6c5882-2e5d-4b55-a028-53ef4e871584 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:01.760472 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6c5882-2e5d-4b55-a028-53ef4e871584 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:01.760792 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6c5882-2e5d-4b55-a028-53ef4e871584 None admin] PATH_INFO: `/v3/projects/b57372dfc633444d9f127657ca57d59c` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:01.765489 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6c5882-2e5d-4b55-a028-53ef4e871584 None admin] RBAC: Authorizing `identity:get_project(project_id=b57372dfc633444d9f127657ca57d59c)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:17:01.766957 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6c5882-2e5d-4b55-a028-53ef4e871584 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:17:01.768611 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 138/293] 10.222.0.30 () {56 vars in 1186 bytes} [Tue Sep 6 04:17:01 2022] GET /identity/v3/projects/b57372dfc633444d9f127657ca57d59c => generated 302 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:17:01.774792 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c5bd89-3977-4c24-98c2-e0938b3bd974 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:01.917921 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c5bd89-3977-4c24-98c2-e0938b3bd974 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:01.939195 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:17:01.957520 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3efe47f-983d-42d1-b19e-9123552228e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:01.958563 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3efe47f-983d-42d1-b19e-9123552228e1 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:01.959051 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3efe47f-983d-42d1-b19e-9123552228e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:01.959553 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3efe47f-983d-42d1-b19e-9123552228e1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:01.960512 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3efe47f-983d-42d1-b19e-9123552228e1 None admin] RBAC: Adding query filter params (role.id=29607b2de39f40d68bb6ceef64b8a732, scope.project.id=b57372dfc633444d9f127657ca57d59c, user.id=35ab231490514d90a4c3fcb3d63e63bd) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:17:01.961036 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3efe47f-983d-42d1-b19e-9123552228e1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:17:01.962716 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3efe47f-983d-42d1-b19e-9123552228e1 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:17:01.969490 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 156/294] 10.222.0.30 () {56 vars in 1374 bytes} [Tue Sep 6 04:17:01 2022] GET /identity/v3/role_assignments?user.id=35ab231490514d90a4c3fcb3d63e63bd&scope.project.id=b57372dfc633444d9f127657ca57d59c&role.id=29607b2de39f40d68bb6ceef64b8a732 => generated 608 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 04:17:02.243572 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f53e16-5498-4d65-9b80-4649bdd29c01 None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:02.243866 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f53e16-5498-4d65-9b80-4649bdd29c01 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:02.244046 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f53e16-5498-4d65-9b80-4649bdd29c01 None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:02.245465 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 139/295] 10.222.0.30 () {54 vars in 927 bytes} [Tue Sep 6 04:17:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 04:17:02.251385 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b425d5cb-b597-47ef-aee0-fdb185de449f None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:02.251986 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b425d5cb-b597-47ef-aee0-fdb185de449f None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:02.252716 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b425d5cb-b597-47ef-aee0-fdb185de449f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:02.274728 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-b425d5cb-b597-47ef-aee0-fdb185de449f None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:17:02.276257 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-b425d5cb-b597-47ef-aee0-fdb185de449f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:17:02.295589 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24a09c1-e27e-4517-8fba-6848608a8912 None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:02.296404 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24a09c1-e27e-4517-8fba-6848608a8912 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:02.296686 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24a09c1-e27e-4517-8fba-6848608a8912 None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:02.298335 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 140/296] 10.222.0.30 () {54 vars in 927 bytes} [Tue Sep 6 04:17:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 06 04:17:02.306612 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978fe560-4c53-4c69-9b18-ba233aaef7b1 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:02.307153 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978fe560-4c53-4c69-9b18-ba233aaef7b1 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:02.307669 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978fe560-4c53-4c69-9b18-ba233aaef7b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:02.329936 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-978fe560-4c53-4c69-9b18-ba233aaef7b1 None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:17:02.331136 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-978fe560-4c53-4c69-9b18-ba233aaef7b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:17:02.376371 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 157/297] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Sep 6 04:17:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 04:17:02.419981 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 141/298] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Sep 6 04:17:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 06 04:17:02.645438 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8b80f8-7217-4cff-b977-26013e3779f4 None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:02.647507 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8b80f8-7217-4cff-b977-26013e3779f4 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:02.648280 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8b80f8-7217-4cff-b977-26013e3779f4 None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:02.650956 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 158/299] 10.222.0.30 () {54 vars in 927 bytes} [Tue Sep 6 04:17:02 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 04:17:02.658990 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219768e8-0346-49b2-a8ba-fc4b6ffefeb5 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:02.660246 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219768e8-0346-49b2-a8ba-fc4b6ffefeb5 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:02.660945 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219768e8-0346-49b2-a8ba-fc4b6ffefeb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:02.673787 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8b80f8-7217-4cff-b977-26013e3779f4 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:02.681209 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-219768e8-0346-49b2-a8ba-fc4b6ffefeb5 None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:17:02.682437 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-219768e8-0346-49b2-a8ba-fc4b6ffefeb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:17:02.734230 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:17:02.781347 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 142/300] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Sep 6 04:17:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 06 04:17:02.781347 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219768e8-0346-49b2-a8ba-fc4b6ffefeb5 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:02.793471 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8b80f8-7217-4cff-b977-26013e3779f4 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:02.809085 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:17:02.841665 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2092f0-06a3-4524-9065-8115811b6537 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:02.842797 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2092f0-06a3-4524-9065-8115811b6537 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:02.843216 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2092f0-06a3-4524-9065-8115811b6537 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:02.843893 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2092f0-06a3-4524-9065-8115811b6537 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:02.845840 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2092f0-06a3-4524-9065-8115811b6537 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:17:02.847683 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2092f0-06a3-4524-9065-8115811b6537 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:17:02.854606 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 159/301] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Sep 6 04:17:02 2022] GET /identity/v3/domains/default => generated 199 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 04:17:02.856206 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2092f0-06a3-4524-9065-8115811b6537 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:02.902326 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219768e8-0346-49b2-a8ba-fc4b6ffefeb5 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:02.945716 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c158828a-50d9-489e-9880-16628b524022 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:02.946358 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c158828a-50d9-489e-9880-16628b524022 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:02.946639 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c158828a-50d9-489e-9880-16628b524022 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:02.946909 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c158828a-50d9-489e-9880-16628b524022 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:02.960758 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c158828a-50d9-489e-9880-16628b524022 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 04:17:02.967203 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-c158828a-50d9-489e-9880-16628b524022 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 06 04:17:02.967203 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 04:17:02.967203 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 04:17:02.967203 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 04:17:02.967203 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 04:17:02.967203 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 04:17:02.967203 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 04:17:02.967203 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 04:17:02.967203 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 04:17:02.967203 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 04:17:02.967203 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 04:17:02.967203 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 04:17:02.967203 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 04:17:02.967203 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 04:17:02.967203 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 04:17:02.967203 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 04:17:02.968109 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 04:17:02.968109 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 04:17:02.968109 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 04:17:02.968109 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 04:17:02.968109 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 04:17:02.968109 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 04:17:02.968109 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 04:17:02.968109 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 04:17:02.968109 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 04:17:02.968109 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 04:17:02.968109 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 04:17:02.968109 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 04:17:02.968109 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 04:17:02.968109 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 04:17:02.968109 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 04:17:02.968109 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 04:17:02.968915 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 04:17:02.968915 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 04:17:02.968915 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 04:17:02.968915 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 04:17:02.968915 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 04:17:02.968915 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 04:17:02.968915 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 04:17:02.968915 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 04:17:02.968915 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 04:17:02.968915 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 04:17:02.968915 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 06 04:17:02.968915 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 04:17:02.968915 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 143/302] 10.222.0.30 () {56 vars in 1099 bytes} [Tue Sep 6 04:17:02 2022] GET /identity/v3/roles/reader => generated 84 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 04:17:02.974404 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c158828a-50d9-489e-9880-16628b524022 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:02.974623 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2092f0-06a3-4524-9065-8115811b6537 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:03.018225 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:17:03.018569 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1aa37ba-251d-4170-a594-79370fdb2a8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:03.019636 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1aa37ba-251d-4170-a594-79370fdb2a8d None admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:03.020048 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1aa37ba-251d-4170-a594-79370fdb2a8d None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:03.020514 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1aa37ba-251d-4170-a594-79370fdb2a8d None admin] PATH_INFO: `/v3/groups` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:03.021970 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1aa37ba-251d-4170-a594-79370fdb2a8d None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:17:03.023581 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1aa37ba-251d-4170-a594-79370fdb2a8d None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:17:03.089379 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 160/303] 10.222.0.30 () {60 vars in 1134 bytes} [Tue Sep 6 04:17:02 2022] POST /identity/v3/groups => generated 229 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 04:17:03.090206 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1aa37ba-251d-4170-a594-79370fdb2a8d None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:03.104518 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c158828a-50d9-489e-9880-16628b524022 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:03.142455 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:17:03.153835 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794b2028-d210-4cf7-bdda-d873952db5e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:03.154390 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794b2028-d210-4cf7-bdda-d873952db5e6 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:03.154670 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794b2028-d210-4cf7-bdda-d873952db5e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:03.154951 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794b2028-d210-4cf7-bdda-d873952db5e6 None admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:03.155635 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794b2028-d210-4cf7-bdda-d873952db5e6 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:17:03.155948 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794b2028-d210-4cf7-bdda-d873952db5e6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:17:03.157411 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794b2028-d210-4cf7-bdda-d873952db5e6 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:17:03.167295 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 144/304] 10.222.0.30 () {56 vars in 1101 bytes} [Tue Sep 6 04:17:02 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:17:03.168624 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09cd58e-af30-45fe-b1e1-bab7307625c9 None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:03.168916 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09cd58e-af30-45fe-b1e1-bab7307625c9 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:03.169190 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09cd58e-af30-45fe-b1e1-bab7307625c9 None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:03.170678 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 145/305] 10.222.0.30 () {54 vars in 927 bytes} [Tue Sep 6 04:17:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 04:17:03.172504 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09cd58e-af30-45fe-b1e1-bab7307625c9 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:03.227015 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1aa37ba-251d-4170-a594-79370fdb2a8d None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:03.268603 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbfd8bd1-cefb-4985-9653-be917bbe87d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:03.268952 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfd8bd1-cefb-4985-9653-be917bbe87d1 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:03.269288 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfd8bd1-cefb-4985-9653-be917bbe87d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:03.269601 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfd8bd1-cefb-4985-9653-be917bbe87d1 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:03.282533 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbfd8bd1-cefb-4985-9653-be917bbe87d1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 04:17:03.284049 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:17:03.288656 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-bbfd8bd1-cefb-4985-9653-be917bbe87d1 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 06 04:17:03.288656 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 04:17:03.288656 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 04:17:03.288656 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 04:17:03.288656 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 04:17:03.288656 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 04:17:03.288656 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 04:17:03.288656 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 04:17:03.288656 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 04:17:03.288656 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 04:17:03.288656 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 04:17:03.288656 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 04:17:03.288656 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 04:17:03.288656 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 04:17:03.288656 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 04:17:03.288656 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 04:17:03.289613 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 04:17:03.289613 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 04:17:03.289613 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 04:17:03.289613 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 04:17:03.289613 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 04:17:03.289613 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 04:17:03.289613 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 04:17:03.289613 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 04:17:03.289613 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 04:17:03.289613 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 04:17:03.289613 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 04:17:03.289613 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 04:17:03.289613 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 04:17:03.289613 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 04:17:03.289613 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 04:17:03.289613 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 04:17:03.290507 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 04:17:03.290507 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 04:17:03.290507 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 04:17:03.290507 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 04:17:03.290507 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 04:17:03.290507 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 04:17:03.290507 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 04:17:03.290507 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 04:17:03.290507 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 04:17:03.290507 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 04:17:03.290507 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 06 04:17:03.290507 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 04:17:03.290507 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 161/306] 10.222.0.30 () {56 vars in 1099 bytes} [Tue Sep 6 04:17:03 2022] GET /identity/v3/roles/member => generated 84 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 04:17:03.291617 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f9da72-4f15-44a8-aba7-d019d662c5bb None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:03.291714 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f9da72-4f15-44a8-aba7-d019d662c5bb None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:03.292007 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f9da72-4f15-44a8-aba7-d019d662c5bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:03.293083 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09cd58e-af30-45fe-b1e1-bab7307625c9 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:03.315576 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-98f9da72-4f15-44a8-aba7-d019d662c5bb None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:17:03.316827 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-98f9da72-4f15-44a8-aba7-d019d662c5bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:17:03.331064 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c80147-47d7-454e-a6ce-cb4c987530f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:03.332019 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c80147-47d7-454e-a6ce-cb4c987530f0 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:03.332487 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c80147-47d7-454e-a6ce-cb4c987530f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:03.332897 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c80147-47d7-454e-a6ce-cb4c987530f0 None admin] PATH_INFO: `/v3/users/53431b281da3464cbd805732b370f432` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:03.353196 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c80147-47d7-454e-a6ce-cb4c987530f0 None admin] RBAC: Authorizing `identity:get_user(user_id=53431b281da3464cbd805732b370f432)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:17:03.355043 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c80147-47d7-454e-a6ce-cb4c987530f0 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:17:03.356870 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 146/307] 10.222.0.30 () {56 vars in 1177 bytes} [Tue Sep 6 04:17:03 2022] GET /identity/v3/users/53431b281da3464cbd805732b370f432 => generated 293 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 04:17:03.357914 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c80147-47d7-454e-a6ce-cb4c987530f0 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:03.397794 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 162/308] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Sep 6 04:17:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 04:17:03.398623 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f9da72-4f15-44a8-aba7-d019d662c5bb None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:03.431363 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:17:03.482076 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c80147-47d7-454e-a6ce-cb4c987530f0 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:03.531952 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe51e57-058c-4688-a2e0-07cd5a772463 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:03.533130 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe51e57-058c-4688-a2e0-07cd5a772463 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:03.533601 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe51e57-058c-4688-a2e0-07cd5a772463 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:03.534192 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe51e57-058c-4688-a2e0-07cd5a772463 None admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:03.535317 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe51e57-058c-4688-a2e0-07cd5a772463 None admin] RBAC: Adding query filter params (name=member) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:17:03.535793 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe51e57-058c-4688-a2e0-07cd5a772463 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:17:03.537949 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe51e57-058c-4688-a2e0-07cd5a772463 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:17:03.549518 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 147/309] 10.222.0.30 () {56 vars in 1101 bytes} [Tue Sep 6 04:17:03 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:17:03.554935 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f9da72-4f15-44a8-aba7-d019d662c5bb None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:03.557397 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe51e57-058c-4688-a2e0-07cd5a772463 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:03.578086 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:17:03.599699 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7a98f5-d7a1-47b7-996d-192de2fa845b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:03.600697 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7a98f5-d7a1-47b7-996d-192de2fa845b None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:03.602412 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7a98f5-d7a1-47b7-996d-192de2fa845b None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:03.602412 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7a98f5-d7a1-47b7-996d-192de2fa845b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:03.602412 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7a98f5-d7a1-47b7-996d-192de2fa845b None admin] RBAC: Adding query filter params (role.id=3c011524d1df425b902b8b32c1b485c3, user.id=53431b281da3464cbd805732b370f432, scope.system=all) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:17:03.603031 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7a98f5-d7a1-47b7-996d-192de2fa845b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:17:03.604862 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7a98f5-d7a1-47b7-996d-192de2fa845b None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:17:03.612442 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 163/310] 10.222.0.30 () {56 vars in 1308 bytes} [Tue Sep 6 04:17:03 2022] GET /identity/v3/role_assignments?user.id=53431b281da3464cbd805732b370f432&scope.system=all&role.id=3c011524d1df425b902b8b32c1b485c3 => generated 226 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:17:03.684429 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe51e57-058c-4688-a2e0-07cd5a772463 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:03.714651 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7a98f5-d7a1-47b7-996d-192de2fa845b None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:03.724301 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b112a9-f751-4324-8bcc-131f646be57b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:03.724917 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b112a9-f751-4324-8bcc-131f646be57b None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:03.725194 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b112a9-f751-4324-8bcc-131f646be57b None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:03.725543 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b112a9-f751-4324-8bcc-131f646be57b None admin] PATH_INFO: `/v3/users/36df010d3b9d4a8e89f1dba7452b7341` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:03.727466 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b112a9-f751-4324-8bcc-131f646be57b None admin] RBAC: Authorizing `identity:get_user(user_id=36df010d3b9d4a8e89f1dba7452b7341)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:17:03.728966 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b112a9-f751-4324-8bcc-131f646be57b None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:17:03.731863 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 148/311] 10.222.0.30 () {56 vars in 1177 bytes} [Tue Sep 6 04:17:03 2022] GET /identity/v3/users/36df010d3b9d4a8e89f1dba7452b7341 => generated 293 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:17:03.737299 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b112a9-f751-4324-8bcc-131f646be57b None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:03.821981 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:17:03.831009 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7a98f5-d7a1-47b7-996d-192de2fa845b None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:03.847794 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:17:03.871926 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86872658-1104-45ed-9ad1-00957a8bb83c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:03.872673 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86872658-1104-45ed-9ad1-00957a8bb83c None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:03.873017 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86872658-1104-45ed-9ad1-00957a8bb83c None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:03.873169 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b112a9-f751-4324-8bcc-131f646be57b None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:03.873442 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86872658-1104-45ed-9ad1-00957a8bb83c None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:03.886401 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86872658-1104-45ed-9ad1-00957a8bb83c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 04:17:03.892672 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-86872658-1104-45ed-9ad1-00957a8bb83c None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 06 04:17:03.892672 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 04:17:03.892672 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 04:17:03.892672 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 04:17:03.892672 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 04:17:03.892672 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 04:17:03.892672 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 04:17:03.892672 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 04:17:03.892672 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 04:17:03.892672 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 04:17:03.892672 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 04:17:03.892672 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 04:17:03.892672 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 04:17:03.892672 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 04:17:03.892672 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 04:17:03.892672 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 04:17:03.893581 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 04:17:03.893581 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 04:17:03.893581 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 04:17:03.893581 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 04:17:03.893581 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 04:17:03.893581 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 04:17:03.893581 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 04:17:03.893581 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 04:17:03.893581 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 04:17:03.893581 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 04:17:03.893581 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 04:17:03.893581 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 04:17:03.893581 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 04:17:03.893581 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 04:17:03.893581 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 04:17:03.893581 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 04:17:03.894395 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 04:17:03.894395 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 04:17:03.894395 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 04:17:03.894395 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 04:17:03.894395 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 04:17:03.894395 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 04:17:03.894395 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 04:17:03.894395 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 04:17:03.894395 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 04:17:03.894395 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 04:17:03.894395 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 06 04:17:03.894395 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 04:17:03.900219 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 164/312] 10.222.0.30 () {56 vars in 1099 bytes} [Tue Sep 6 04:17:03 2022] GET /identity/v3/roles/reader => generated 84 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 04:17:03.901300 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86872658-1104-45ed-9ad1-00957a8bb83c None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:03.918690 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2260aba7-7dc3-49c2-8373-05d1bcbc05b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:03.919346 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2260aba7-7dc3-49c2-8373-05d1bcbc05b8 None admin] REQUEST_METHOD: `PUT` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:03.919628 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2260aba7-7dc3-49c2-8373-05d1bcbc05b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:03.920193 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2260aba7-7dc3-49c2-8373-05d1bcbc05b8 None admin] PATH_INFO: `/v3/system/users/36df010d3b9d4a8e89f1dba7452b7341/roles/29607b2de39f40d68bb6ceef64b8a732` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:03.922412 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2260aba7-7dc3-49c2-8373-05d1bcbc05b8 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=36df010d3b9d4a8e89f1dba7452b7341, role_id=29607b2de39f40d68bb6ceef64b8a732)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:17:03.924570 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2260aba7-7dc3-49c2-8373-05d1bcbc05b8 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:17:04.017192 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 149/313] 10.222.0.30 () {58 vars in 1334 bytes} [Tue Sep 6 04:17:03 2022] PUT /identity/v3/system/users/36df010d3b9d4a8e89f1dba7452b7341/roles/29607b2de39f40d68bb6ceef64b8a732 => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 04:17:04.036612 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86872658-1104-45ed-9ad1-00957a8bb83c None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:04.078798 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5ecf86-88fd-4513-9436-13e8f7ca69bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:04.079435 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5ecf86-88fd-4513-9436-13e8f7ca69bf None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:04.079717 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5ecf86-88fd-4513-9436-13e8f7ca69bf None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:04.079992 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5ecf86-88fd-4513-9436-13e8f7ca69bf None admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:04.080682 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5ecf86-88fd-4513-9436-13e8f7ca69bf None admin] RBAC: Adding query filter params (name=reader) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:17:04.081045 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5ecf86-88fd-4513-9436-13e8f7ca69bf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:17:04.082634 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5ecf86-88fd-4513-9436-13e8f7ca69bf None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:17:04.093477 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 165/314] 10.222.0.30 () {56 vars in 1101 bytes} [Tue Sep 6 04:17:03 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:17:04.099145 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2260aba7-7dc3-49c2-8373-05d1bcbc05b8 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:04.214706 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2260aba7-7dc3-49c2-8373-05d1bcbc05b8 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:04.252633 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc4cafd-a290-4d7f-8f74-c65b040e9dfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:04.253350 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc4cafd-a290-4d7f-8f74-c65b040e9dfd None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:04.253636 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc4cafd-a290-4d7f-8f74-c65b040e9dfd None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:04.253907 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc4cafd-a290-4d7f-8f74-c65b040e9dfd None admin] PATH_INFO: `/v3/users/472fdc6775594fa3a9c9a96577625ace` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:04.255810 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc4cafd-a290-4d7f-8f74-c65b040e9dfd None admin] RBAC: Authorizing `identity:get_user(user_id=472fdc6775594fa3a9c9a96577625ace)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:17:04.257334 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc4cafd-a290-4d7f-8f74-c65b040e9dfd None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:17:04.259264 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 150/315] 10.222.0.30 () {56 vars in 1177 bytes} [Tue Sep 6 04:17:04 2022] GET /identity/v3/users/472fdc6775594fa3a9c9a96577625ace => generated 297 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 04:17:04.264838 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5ecf86-88fd-4513-9436-13e8f7ca69bf None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:04.282973 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:17:04.366625 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5ecf86-88fd-4513-9436-13e8f7ca69bf None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:04.413107 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1364ca7-5cdf-4ca5-ac41-0b31d1bd0331 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:04.413763 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1364ca7-5cdf-4ca5-ac41-0b31d1bd0331 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:04.414044 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1364ca7-5cdf-4ca5-ac41-0b31d1bd0331 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:04.414377 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1364ca7-5cdf-4ca5-ac41-0b31d1bd0331 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:04.488428 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1364ca7-5cdf-4ca5-ac41-0b31d1bd0331 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:17:04.490911 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1364ca7-5cdf-4ca5-ac41-0b31d1bd0331 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:17:04.509660 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:17:04.597104 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 166/316] 10.222.0.30 () {58 vars in 1282 bytes} [Tue Sep 6 04:17:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 06 04:17:04.603471 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc4cafd-a290-4d7f-8f74-c65b040e9dfd None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:04.620608 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:17:04.717527 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc4cafd-a290-4d7f-8f74-c65b040e9dfd None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:04.763125 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3955491-df12-4d6c-928b-2bab31aae5c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:04.763789 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3955491-df12-4d6c-928b-2bab31aae5c6 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:04.764070 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3955491-df12-4d6c-928b-2bab31aae5c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:04.764342 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3955491-df12-4d6c-928b-2bab31aae5c6 None admin] PATH_INFO: `/v3/projects/b57372dfc633444d9f127657ca57d59c` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:04.765542 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3955491-df12-4d6c-928b-2bab31aae5c6 None admin] RBAC: Authorizing `identity:get_project(project_id=b57372dfc633444d9f127657ca57d59c)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:17:04.766991 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3955491-df12-4d6c-928b-2bab31aae5c6 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:17:04.768808 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 151/317] 10.222.0.30 () {56 vars in 1186 bytes} [Tue Sep 6 04:17:04 2022] GET /identity/v3/projects/b57372dfc633444d9f127657ca57d59c => generated 302 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:17:04.775417 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1364ca7-5cdf-4ca5-ac41-0b31d1bd0331 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:04.886641 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1364ca7-5cdf-4ca5-ac41-0b31d1bd0331 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:04.905495 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:17:04.925861 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e45e52f-70bc-4b18-bbfe-793e399d960b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:04.926470 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e45e52f-70bc-4b18-bbfe-793e399d960b None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:04.926767 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e45e52f-70bc-4b18-bbfe-793e399d960b None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:04.927191 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e45e52f-70bc-4b18-bbfe-793e399d960b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:04.928067 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e45e52f-70bc-4b18-bbfe-793e399d960b None admin] RBAC: Adding query filter params (role.id=3c011524d1df425b902b8b32c1b485c3, scope.project.id=b57372dfc633444d9f127657ca57d59c, user.id=472fdc6775594fa3a9c9a96577625ace) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:17:04.928379 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e45e52f-70bc-4b18-bbfe-793e399d960b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:17:04.929861 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e45e52f-70bc-4b18-bbfe-793e399d960b None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:17:04.935820 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 167/318] 10.222.0.30 () {56 vars in 1374 bytes} [Tue Sep 6 04:17:04 2022] GET /identity/v3/role_assignments?user.id=472fdc6775594fa3a9c9a96577625ace&scope.project.id=b57372dfc633444d9f127657ca57d59c&role.id=3c011524d1df425b902b8b32c1b485c3 => generated 608 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:17:05.109010 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e249b7f-4d92-4908-a89a-1f3d35e30895 None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:05.109375 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e249b7f-4d92-4908-a89a-1f3d35e30895 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:05.109592 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e249b7f-4d92-4908-a89a-1f3d35e30895 None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:05.111138 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 152/319] 10.222.0.30 () {54 vars in 927 bytes} [Tue Sep 6 04:17:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 04:17:05.116553 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0591722-372f-483d-8f07-9d93bfb3cc41 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:05.116841 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0591722-372f-483d-8f07-9d93bfb3cc41 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:05.117116 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0591722-372f-483d-8f07-9d93bfb3cc41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:05.138640 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-d0591722-372f-483d-8f07-9d93bfb3cc41 None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:17:05.140136 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-d0591722-372f-483d-8f07-9d93bfb3cc41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:17:05.212791 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 168/320] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Sep 6 04:17:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 04:17:05.487152 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e249b7f-4d92-4908-a89a-1f3d35e30895 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:05.575544 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:17:05.595014 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e249b7f-4d92-4908-a89a-1f3d35e30895 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:05.631173 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47cab9ea-19ff-4331-aab6-c88893a327c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:05.633428 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cab9ea-19ff-4331-aab6-c88893a327c4 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:05.633428 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cab9ea-19ff-4331-aab6-c88893a327c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:05.633428 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cab9ea-19ff-4331-aab6-c88893a327c4 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:05.633661 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47cab9ea-19ff-4331-aab6-c88893a327c4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:17:05.635082 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47cab9ea-19ff-4331-aab6-c88893a327c4 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:17:05.636827 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 153/321] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Sep 6 04:17:05 2022] GET /identity/v3/domains/default => generated 199 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:17:05.642300 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0591722-372f-483d-8f07-9d93bfb3cc41 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:05.707246 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03091bca-a61c-45fd-8c7f-8e197e852490 None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:05.707530 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03091bca-a61c-45fd-8c7f-8e197e852490 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:05.707827 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03091bca-a61c-45fd-8c7f-8e197e852490 None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:05.710319 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 154/322] 10.222.0.30 () {54 vars in 927 bytes} [Tue Sep 6 04:17:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 06 04:17:05.715153 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc91dd82-6783-424c-baa6-60ed45872df6 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:05.715438 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc91dd82-6783-424c-baa6-60ed45872df6 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:05.715716 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc91dd82-6783-424c-baa6-60ed45872df6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:05.733981 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-cc91dd82-6783-424c-baa6-60ed45872df6 None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:17:05.735174 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-cc91dd82-6783-424c-baa6-60ed45872df6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:17:05.748752 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0591722-372f-483d-8f07-9d93bfb3cc41 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:05.760256 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:17:05.788936 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b51f087-d0ba-40ff-99c6-bbb798e14263 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:05.789606 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b51f087-d0ba-40ff-99c6-bbb798e14263 None admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:05.789936 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b51f087-d0ba-40ff-99c6-bbb798e14263 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:05.790216 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b51f087-d0ba-40ff-99c6-bbb798e14263 None admin] PATH_INFO: `/v3/groups` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:05.790863 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b51f087-d0ba-40ff-99c6-bbb798e14263 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:17:05.792325 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b51f087-d0ba-40ff-99c6-bbb798e14263 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:17:05.810293 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 155/323] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Sep 6 04:17:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 06 04:17:05.888687 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 169/324] 10.222.0.30 () {60 vars in 1134 bytes} [Tue Sep 6 04:17:05 2022] POST /identity/v3/groups => generated 226 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 04:17:06.084882 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc91dd82-6783-424c-baa6-60ed45872df6 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:06.096519 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32dd75da-2f20-4036-b6cf-40b660f31e5a None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:06.096827 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32dd75da-2f20-4036-b6cf-40b660f31e5a None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:06.097079 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32dd75da-2f20-4036-b6cf-40b660f31e5a None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:06.098890 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 170/325] 10.222.0.30 () {54 vars in 927 bytes} [Tue Sep 6 04:17:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 06 04:17:06.105276 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da8a6db-f2de-4ff0-b5d1-efce4bb7f5a6 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:06.105595 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da8a6db-f2de-4ff0-b5d1-efce4bb7f5a6 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:06.105868 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da8a6db-f2de-4ff0-b5d1-efce4bb7f5a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:06.127845 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-5da8a6db-f2de-4ff0-b5d1-efce4bb7f5a6 None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:17:06.129075 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-5da8a6db-f2de-4ff0-b5d1-efce4bb7f5a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:17:06.183997 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:17:06.206208 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc91dd82-6783-424c-baa6-60ed45872df6 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:06.220638 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 171/326] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Sep 6 04:17:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 06 04:17:06.249093 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7227c9d-3fac-41c5-92c2-b5f3514fd6de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:06.249767 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7227c9d-3fac-41c5-92c2-b5f3514fd6de None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:06.250047 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7227c9d-3fac-41c5-92c2-b5f3514fd6de None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:06.250315 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7227c9d-3fac-41c5-92c2-b5f3514fd6de None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:06.262547 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7227c9d-3fac-41c5-92c2-b5f3514fd6de None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 04:17:06.267466 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-a7227c9d-3fac-41c5-92c2-b5f3514fd6de None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 06 04:17:06.267466 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 04:17:06.267466 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 04:17:06.267466 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 04:17:06.267466 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 04:17:06.267466 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 04:17:06.267466 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 04:17:06.267466 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 04:17:06.267466 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 04:17:06.267466 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 04:17:06.267466 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 04:17:06.267466 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 04:17:06.267466 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 04:17:06.267466 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 04:17:06.267466 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 04:17:06.267466 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 04:17:06.268339 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 04:17:06.268339 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 04:17:06.268339 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 04:17:06.268339 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 04:17:06.268339 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 04:17:06.268339 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 04:17:06.268339 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 04:17:06.268339 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 04:17:06.268339 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 04:17:06.268339 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 04:17:06.268339 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 04:17:06.268339 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 04:17:06.268339 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 04:17:06.268339 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 04:17:06.268339 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 04:17:06.268339 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 04:17:06.269121 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 04:17:06.269121 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 04:17:06.269121 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 04:17:06.269121 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 04:17:06.269121 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 04:17:06.269121 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 04:17:06.269121 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 04:17:06.269121 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 04:17:06.269121 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 04:17:06.269121 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 04:17:06.269121 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 06 04:17:06.269121 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 04:17:06.269121 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 156/327] 10.222.0.30 () {56 vars in 1099 bytes} [Tue Sep 6 04:17:06 2022] GET /identity/v3/roles/reader => generated 84 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 04:17:06.274696 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da8a6db-f2de-4ff0-b5d1-efce4bb7f5a6 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:06.393182 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:17:06.397565 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da8a6db-f2de-4ff0-b5d1-efce4bb7f5a6 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:06.433856 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071d8dd8-73ae-435c-9851-812266571bb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:06.434480 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071d8dd8-73ae-435c-9851-812266571bb9 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:06.434763 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071d8dd8-73ae-435c-9851-812266571bb9 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:06.435037 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071d8dd8-73ae-435c-9851-812266571bb9 None admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:06.435715 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-071d8dd8-73ae-435c-9851-812266571bb9 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:17:06.436023 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-071d8dd8-73ae-435c-9851-812266571bb9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:17:06.437508 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-071d8dd8-73ae-435c-9851-812266571bb9 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:17:06.447663 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 172/328] 10.222.0.30 () {56 vars in 1101 bytes} [Tue Sep 6 04:17:06 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:17:06.453182 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7227c9d-3fac-41c5-92c2-b5f3514fd6de None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:06.497550 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071d8dd8-73ae-435c-9851-812266571bb9 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:06.582573 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7227c9d-3fac-41c5-92c2-b5f3514fd6de None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:06.632973 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071d8dd8-73ae-435c-9851-812266571bb9 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:06.635418 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1daee27-d21d-46fd-b032-e7336d858678 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:06.636905 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1daee27-d21d-46fd-b032-e7336d858678 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:06.637190 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1daee27-d21d-46fd-b032-e7336d858678 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:06.637508 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1daee27-d21d-46fd-b032-e7336d858678 None admin] PATH_INFO: `/v3/users/53431b281da3464cbd805732b370f432` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:06.639125 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1daee27-d21d-46fd-b032-e7336d858678 None admin] RBAC: Authorizing `identity:get_user(user_id=53431b281da3464cbd805732b370f432)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:17:06.640241 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1daee27-d21d-46fd-b032-e7336d858678 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:17:06.642178 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 157/329] 10.222.0.30 () {56 vars in 1177 bytes} [Tue Sep 6 04:17:06 2022] GET /identity/v3/users/53431b281da3464cbd805732b370f432 => generated 293 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:17:06.648647 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1daee27-d21d-46fd-b032-e7336d858678 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:06.666860 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:17:06.687372 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9c0c8a-1275-44a6-8241-0445503ba3d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:06.688020 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9c0c8a-1275-44a6-8241-0445503ba3d5 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:06.688293 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9c0c8a-1275-44a6-8241-0445503ba3d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:06.688558 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9c0c8a-1275-44a6-8241-0445503ba3d5 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:06.714152 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9c0c8a-1275-44a6-8241-0445503ba3d5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 04:17:06.721126 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-5f9c0c8a-1275-44a6-8241-0445503ba3d5 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 06 04:17:06.721126 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 04:17:06.721126 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 04:17:06.721126 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 04:17:06.721126 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 04:17:06.721126 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 04:17:06.721126 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 04:17:06.721126 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 04:17:06.721126 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 04:17:06.721126 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 04:17:06.721126 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 04:17:06.721126 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 04:17:06.721126 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 04:17:06.721126 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 04:17:06.721126 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 04:17:06.721126 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 04:17:06.722063 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 04:17:06.722063 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 04:17:06.722063 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 04:17:06.722063 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 04:17:06.722063 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 04:17:06.722063 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 04:17:06.722063 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 04:17:06.722063 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 04:17:06.722063 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 04:17:06.722063 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 04:17:06.722063 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 04:17:06.722063 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 04:17:06.722063 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 04:17:06.722063 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 04:17:06.722063 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 04:17:06.722063 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 04:17:06.722873 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 04:17:06.722873 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 04:17:06.722873 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 04:17:06.722873 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 04:17:06.722873 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 04:17:06.722873 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 04:17:06.722873 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 04:17:06.722873 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 04:17:06.722873 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 04:17:06.722873 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 04:17:06.722873 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 06 04:17:06.722873 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 04:17:06.722873 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 173/330] 10.222.0.30 () {56 vars in 1099 bytes} [Tue Sep 6 04:17:06 2022] GET /identity/v3/roles/member => generated 84 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 04:17:06.729974 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9c0c8a-1275-44a6-8241-0445503ba3d5 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:06.761941 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:17:06.787494 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1daee27-d21d-46fd-b032-e7336d858678 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:06.827094 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00326bd9-2d4a-4f67-b994-10d0db1a8473 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:06.828037 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00326bd9-2d4a-4f67-b994-10d0db1a8473 None admin] REQUEST_METHOD: `PUT` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:06.828464 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00326bd9-2d4a-4f67-b994-10d0db1a8473 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:06.828886 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00326bd9-2d4a-4f67-b994-10d0db1a8473 None admin] PATH_INFO: `/v3/system/users/53431b281da3464cbd805732b370f432/roles/3c011524d1df425b902b8b32c1b485c3` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:06.830537 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00326bd9-2d4a-4f67-b994-10d0db1a8473 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=53431b281da3464cbd805732b370f432, role_id=3c011524d1df425b902b8b32c1b485c3)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:17:06.832070 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00326bd9-2d4a-4f67-b994-10d0db1a8473 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:17:06.870674 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9c0c8a-1275-44a6-8241-0445503ba3d5 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:06.897564 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 158/331] 10.222.0.30 () {58 vars in 1334 bytes} [Tue Sep 6 04:17:06 2022] PUT /identity/v3/system/users/53431b281da3464cbd805732b370f432/roles/3c011524d1df425b902b8b32c1b485c3 => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 04:17:06.919521 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80beb28e-ee80-4379-b2f8-4185cf35ea1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:06.920509 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80beb28e-ee80-4379-b2f8-4185cf35ea1b None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:06.920997 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80beb28e-ee80-4379-b2f8-4185cf35ea1b None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:06.921501 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80beb28e-ee80-4379-b2f8-4185cf35ea1b None admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:06.922377 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80beb28e-ee80-4379-b2f8-4185cf35ea1b None admin] RBAC: Adding query filter params (name=member) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:17:06.922863 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80beb28e-ee80-4379-b2f8-4185cf35ea1b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:17:06.924491 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80beb28e-ee80-4379-b2f8-4185cf35ea1b None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:17:06.935938 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 174/332] 10.222.0.30 () {56 vars in 1101 bytes} [Tue Sep 6 04:17:06 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 04:17:06.941397 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00326bd9-2d4a-4f67-b994-10d0db1a8473 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:07.023942 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:17:07.059504 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00326bd9-2d4a-4f67-b994-10d0db1a8473 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:07.104768 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeaa6a28-bea6-4784-87e8-73fefe8cf6a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:07.105907 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeaa6a28-bea6-4784-87e8-73fefe8cf6a4 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:07.106395 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeaa6a28-bea6-4784-87e8-73fefe8cf6a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:07.106848 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeaa6a28-bea6-4784-87e8-73fefe8cf6a4 None admin] PATH_INFO: `/v3/users/36df010d3b9d4a8e89f1dba7452b7341` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:07.110076 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeaa6a28-bea6-4784-87e8-73fefe8cf6a4 None admin] RBAC: Authorizing `identity:get_user(user_id=36df010d3b9d4a8e89f1dba7452b7341)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:17:07.111535 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeaa6a28-bea6-4784-87e8-73fefe8cf6a4 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:17:07.113523 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 159/333] 10.222.0.30 () {56 vars in 1177 bytes} [Tue Sep 6 04:17:06 2022] GET /identity/v3/users/36df010d3b9d4a8e89f1dba7452b7341 => generated 293 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:17:07.119940 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80beb28e-ee80-4379-b2f8-4185cf35ea1b None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:07.251814 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:17:07.270190 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80beb28e-ee80-4379-b2f8-4185cf35ea1b None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:07.308597 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64226831-26d0-4743-b797-1a6ce250f545 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:07.309211 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64226831-26d0-4743-b797-1a6ce250f545 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:07.309521 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64226831-26d0-4743-b797-1a6ce250f545 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:07.309793 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64226831-26d0-4743-b797-1a6ce250f545 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:07.310523 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64226831-26d0-4743-b797-1a6ce250f545 None admin] RBAC: Adding query filter params (role.id=29607b2de39f40d68bb6ceef64b8a732, user.id=36df010d3b9d4a8e89f1dba7452b7341, scope.system=all) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:17:07.310832 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64226831-26d0-4743-b797-1a6ce250f545 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:17:07.312270 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64226831-26d0-4743-b797-1a6ce250f545 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:17:07.324760 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 175/334] 10.222.0.30 () {56 vars in 1308 bytes} [Tue Sep 6 04:17:07 2022] GET /identity/v3/role_assignments?user.id=36df010d3b9d4a8e89f1dba7452b7341&scope.system=all&role.id=29607b2de39f40d68bb6ceef64b8a732 => generated 510 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:17:08.072218 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229ab57e-2388-4e89-9162-e295a5a231e5 None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:08.072727 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229ab57e-2388-4e89-9162-e295a5a231e5 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:08.072803 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229ab57e-2388-4e89-9162-e295a5a231e5 None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:08.074418 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 160/335] 10.222.0.30 () {54 vars in 927 bytes} [Tue Sep 6 04:17:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 04:17:08.080392 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dca0f32-afc2-4eea-aeb3-f12ea166d407 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:08.080796 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dca0f32-afc2-4eea-aeb3-f12ea166d407 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:08.080965 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dca0f32-afc2-4eea-aeb3-f12ea166d407 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:08.102528 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-3dca0f32-afc2-4eea-aeb3-f12ea166d407 None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:17:08.103737 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-3dca0f32-afc2-4eea-aeb3-f12ea166d407 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:17:08.180730 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 176/336] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Sep 6 04:17:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 06 04:17:08.663717 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229ab57e-2388-4e89-9162-e295a5a231e5 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:08.793256 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229ab57e-2388-4e89-9162-e295a5a231e5 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:08.836431 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2842345-b663-4f92-ad8e-53ffef96e3f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:08.837085 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2842345-b663-4f92-ad8e-53ffef96e3f2 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:08.837398 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2842345-b663-4f92-ad8e-53ffef96e3f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:08.837678 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2842345-b663-4f92-ad8e-53ffef96e3f2 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:08.848108 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:17:08.853257 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2842345-b663-4f92-ad8e-53ffef96e3f2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 04:17:08.858168 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-d2842345-b663-4f92-ad8e-53ffef96e3f2 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 06 04:17:08.858168 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 04:17:08.858168 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 04:17:08.858168 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 04:17:08.858168 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 04:17:08.858168 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 04:17:08.858168 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 04:17:08.858168 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 04:17:08.858168 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 04:17:08.858168 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 04:17:08.858168 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 04:17:08.858168 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 04:17:08.858168 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 04:17:08.858168 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 04:17:08.858168 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 04:17:08.858168 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 04:17:08.861066 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 04:17:08.861066 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 04:17:08.861066 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 04:17:08.861066 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 04:17:08.861066 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 04:17:08.861066 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 04:17:08.861066 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 04:17:08.861066 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 04:17:08.861066 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 04:17:08.861066 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 04:17:08.861066 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 04:17:08.861066 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 04:17:08.861066 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 04:17:08.861066 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 04:17:08.861066 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 04:17:08.861066 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 04:17:08.862011 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 04:17:08.862011 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 04:17:08.862011 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 04:17:08.862011 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 04:17:08.862011 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 04:17:08.862011 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 04:17:08.862011 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 04:17:08.862011 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 04:17:08.862011 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 04:17:08.862011 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 04:17:08.862011 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 06 04:17:08.862011 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 04:17:08.862011 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 161/337] 10.222.0.30 () {56 vars in 1099 bytes} [Tue Sep 6 04:17:08 2022] GET /identity/v3/roles/member => generated 84 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 04:17:08.866022 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dca0f32-afc2-4eea-aeb3-f12ea166d407 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:08.945233 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:17:08.977771 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dca0f32-afc2-4eea-aeb3-f12ea166d407 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:09.017023 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8563c2ee-28cc-483b-9022-27398a500362 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:09.017665 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8563c2ee-28cc-483b-9022-27398a500362 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:09.017949 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8563c2ee-28cc-483b-9022-27398a500362 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:09.018222 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8563c2ee-28cc-483b-9022-27398a500362 None admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:09.018900 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8563c2ee-28cc-483b-9022-27398a500362 None admin] RBAC: Adding query filter params (name=member) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:17:09.019203 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8563c2ee-28cc-483b-9022-27398a500362 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:17:09.021131 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8563c2ee-28cc-483b-9022-27398a500362 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:17:09.029908 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 177/338] 10.222.0.30 () {56 vars in 1101 bytes} [Tue Sep 6 04:17:08 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:17:09.035531 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2842345-b663-4f92-ad8e-53ffef96e3f2 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:09.155767 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2842345-b663-4f92-ad8e-53ffef96e3f2 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:09.196024 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125295e3-af03-46c2-a816-00391861ca56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:09.196571 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125295e3-af03-46c2-a816-00391861ca56 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:09.196856 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125295e3-af03-46c2-a816-00391861ca56 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:09.197130 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125295e3-af03-46c2-a816-00391861ca56 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:09.232841 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7a3d63-6f99-4823-b5b7-84388a932a55 None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:09.233354 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7a3d63-6f99-4823-b5b7-84388a932a55 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:09.233431 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7a3d63-6f99-4823-b5b7-84388a932a55 None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:09.234931 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 178/339] 10.222.0.30 () {54 vars in 927 bytes} [Tue Sep 6 04:17:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 04:17:09.243891 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a7ba50-b941-4be7-8489-f6e0acfad510 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:09.244164 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a7ba50-b941-4be7-8489-f6e0acfad510 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:09.244444 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a7ba50-b941-4be7-8489-f6e0acfad510 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:09.273147 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-b6a7ba50-b941-4be7-8489-f6e0acfad510 None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:17:09.279760 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-b6a7ba50-b941-4be7-8489-f6e0acfad510 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:17:09.326247 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:17:09.329299 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125295e3-af03-46c2-a816-00391861ca56 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:17:09.330829 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125295e3-af03-46c2-a816-00391861ca56 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:17:09.413123 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 179/340] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Sep 6 04:17:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 04:17:09.428338 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6b3b12-7711-4bdb-80b3-97316a9f52bf None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:09.428630 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6b3b12-7711-4bdb-80b3-97316a9f52bf None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:09.428906 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6b3b12-7711-4bdb-80b3-97316a9f52bf None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:09.430551 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 180/341] 10.222.0.30 () {54 vars in 927 bytes} [Tue Sep 6 04:17:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 04:17:09.443650 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1cee37-0ffb-476a-88f2-5cd7faa9d887 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:09.443965 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1cee37-0ffb-476a-88f2-5cd7faa9d887 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:09.446965 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1cee37-0ffb-476a-88f2-5cd7faa9d887 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:09.480878 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 162/342] 10.222.0.30 () {58 vars in 1282 bytes} [Tue Sep 6 04:17:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 446 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 06 04:17:09.485150 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-8b1cee37-0ffb-476a-88f2-5cd7faa9d887 None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:17:09.485150 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-8b1cee37-0ffb-476a-88f2-5cd7faa9d887 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:17:09.486750 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125295e3-af03-46c2-a816-00391861ca56 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:09.610414 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:17:09.635997 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 181/343] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Sep 6 04:17:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 06 04:17:09.652111 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:17:09.655080 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125295e3-af03-46c2-a816-00391861ca56 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:09.734124 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02dd8519-c705-401c-a86b-c8a1a90e7759 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:09.734685 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02dd8519-c705-401c-a86b-c8a1a90e7759 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:09.734971 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02dd8519-c705-401c-a86b-c8a1a90e7759 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:09.735247 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02dd8519-c705-401c-a86b-c8a1a90e7759 None admin] PATH_INFO: `/v3/projects/a88c6977d0554a1eab3e30118ede2f46` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:09.736476 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02dd8519-c705-401c-a86b-c8a1a90e7759 None admin] RBAC: Authorizing `identity:get_project(project_id=a88c6977d0554a1eab3e30118ede2f46)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:17:09.737946 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02dd8519-c705-401c-a86b-c8a1a90e7759 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:17:09.740433 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 163/344] 10.222.0.30 () {56 vars in 1186 bytes} [Tue Sep 6 04:17:09 2022] GET /identity/v3/projects/a88c6977d0554a1eab3e30118ede2f46 => generated 298 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:17:09.747080 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1cee37-0ffb-476a-88f2-5cd7faa9d887 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:09.889252 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1cee37-0ffb-476a-88f2-5cd7faa9d887 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:09.902822 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02dd8519-c705-401c-a86b-c8a1a90e7759 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:09.934245 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa6d840-57db-4037-98ab-9ba874006ce8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:09.936224 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa6d840-57db-4037-98ab-9ba874006ce8 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:09.936512 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa6d840-57db-4037-98ab-9ba874006ce8 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:09.936791 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa6d840-57db-4037-98ab-9ba874006ce8 None admin] PATH_INFO: `/v3/groups/60fbab6f26044826b072daa5861abbb4` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:09.946885 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa6d840-57db-4037-98ab-9ba874006ce8 None admin] RBAC: Authorizing `identity:get_group(group_id=60fbab6f26044826b072daa5861abbb4)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:17:09.951098 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa6d840-57db-4037-98ab-9ba874006ce8 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:17:09.958574 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 182/345] 10.222.0.30 () {56 vars in 1180 bytes} [Tue Sep 6 04:17:09 2022] GET /identity/v3/groups/60fbab6f26044826b072daa5861abbb4 => generated 226 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:17:09.960540 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa6d840-57db-4037-98ab-9ba874006ce8 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:10.055122 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:17:10.073449 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02dd8519-c705-401c-a86b-c8a1a90e7759 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:10.109250 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e5e8e6-38aa-4728-a2c0-ee018839ff3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:10.110416 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e5e8e6-38aa-4728-a2c0-ee018839ff3d None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:10.110735 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e5e8e6-38aa-4728-a2c0-ee018839ff3d None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:10.111120 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e5e8e6-38aa-4728-a2c0-ee018839ff3d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:10.123615 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa6d840-57db-4037-98ab-9ba874006ce8 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:10.129234 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e5e8e6-38aa-4728-a2c0-ee018839ff3d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 04:17:10.137356 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-b6e5e8e6-38aa-4728-a2c0-ee018839ff3d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 06 04:17:10.137356 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 04:17:10.137356 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 04:17:10.137356 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 04:17:10.137356 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 04:17:10.137356 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 04:17:10.137356 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 04:17:10.137356 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 04:17:10.137356 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 04:17:10.137356 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 04:17:10.137356 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 04:17:10.137356 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 04:17:10.137356 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 04:17:10.137356 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 04:17:10.137356 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 04:17:10.137356 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 04:17:10.138315 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 04:17:10.138315 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 04:17:10.138315 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 04:17:10.138315 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 04:17:10.138315 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 04:17:10.138315 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 04:17:10.138315 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 04:17:10.138315 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 04:17:10.138315 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 04:17:10.138315 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 04:17:10.138315 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 04:17:10.138315 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 04:17:10.138315 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 04:17:10.138315 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 04:17:10.138315 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 04:17:10.138315 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 04:17:10.139105 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 04:17:10.139105 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 04:17:10.139105 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 04:17:10.139105 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 04:17:10.139105 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 04:17:10.139105 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 04:17:10.139105 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 04:17:10.139105 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 04:17:10.139105 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 04:17:10.139105 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 04:17:10.139105 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 06 04:17:10.139105 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 04:17:10.139105 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 164/346] 10.222.0.30 () {56 vars in 1114 bytes} [Tue Sep 6 04:17:09 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 04:17:10.156914 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e5e8e6-38aa-4728-a2c0-ee018839ff3d None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:10.186234 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12fa590a-b588-4959-93c7-0f4da703b6cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:10.187112 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fa590a-b588-4959-93c7-0f4da703b6cf None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:10.187525 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fa590a-b588-4959-93c7-0f4da703b6cf None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:10.187925 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fa590a-b588-4959-93c7-0f4da703b6cf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:10.188848 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12fa590a-b588-4959-93c7-0f4da703b6cf None admin] RBAC: Adding query filter params (group.id=60fbab6f26044826b072daa5861abbb4, role.id=29607b2de39f40d68bb6ceef64b8a732, scope.project.id=a88c6977d0554a1eab3e30118ede2f46) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:17:10.189292 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12fa590a-b588-4959-93c7-0f4da703b6cf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:17:10.190957 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12fa590a-b588-4959-93c7-0f4da703b6cf None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:17:10.199140 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 183/347] 10.222.0.30 () {56 vars in 1376 bytes} [Tue Sep 6 04:17:09 2022] GET /identity/v3/role_assignments?group.id=60fbab6f26044826b072daa5861abbb4&scope.project.id=a88c6977d0554a1eab3e30118ede2f46&role.id=29607b2de39f40d68bb6ceef64b8a732 => generated 260 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:17:10.213673 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26fda70-f7a0-46c9-8423-bdab19a5b43b None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:10.214112 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26fda70-f7a0-46c9-8423-bdab19a5b43b None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:10.214482 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26fda70-f7a0-46c9-8423-bdab19a5b43b None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:10.217483 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 184/348] 10.222.0.30 () {54 vars in 927 bytes} [Tue Sep 6 04:17:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 06 04:17:10.222402 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77a75db-0dc8-498e-955f-77828634061f None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:10.222933 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77a75db-0dc8-498e-955f-77828634061f None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:10.223217 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77a75db-0dc8-498e-955f-77828634061f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:10.258464 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:17:10.262707 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-d77a75db-0dc8-498e-955f-77828634061f None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:17:10.264118 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-d77a75db-0dc8-498e-955f-77828634061f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:17:10.358986 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 185/349] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Sep 6 04:17:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 06 04:17:10.359885 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77a75db-0dc8-498e-955f-77828634061f None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:10.438193 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:17:10.482532 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e5e8e6-38aa-4728-a2c0-ee018839ff3d None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:10.494253 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77a75db-0dc8-498e-955f-77828634061f None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:10.538656 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257302d7-d5ad-4ca7-b285-1ab5fee3448a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:10.539551 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257302d7-d5ad-4ca7-b285-1ab5fee3448a None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:10.539837 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257302d7-d5ad-4ca7-b285-1ab5fee3448a None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:10.540107 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257302d7-d5ad-4ca7-b285-1ab5fee3448a None admin] PATH_INFO: `/v3/roles/member` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:10.551460 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-257302d7-d5ad-4ca7-b285-1ab5fee3448a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 04:17:10.554957 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244251bb-4afc-49db-af03-55c1b9685b42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:10.555588 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244251bb-4afc-49db-af03-55c1b9685b42 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:10.555868 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244251bb-4afc-49db-af03-55c1b9685b42 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:10.556140 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244251bb-4afc-49db-af03-55c1b9685b42 None admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:10.556813 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244251bb-4afc-49db-af03-55c1b9685b42 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:17:10.557118 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244251bb-4afc-49db-af03-55c1b9685b42 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:17:10.557456 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-257302d7-d5ad-4ca7-b285-1ab5fee3448a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 06 04:17:10.557456 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 04:17:10.557456 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 04:17:10.557456 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 04:17:10.557456 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 04:17:10.557456 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 04:17:10.557456 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 04:17:10.557456 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 04:17:10.557456 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 04:17:10.557456 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 04:17:10.557456 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 04:17:10.557456 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 04:17:10.557456 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 04:17:10.557456 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 04:17:10.557456 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 04:17:10.557456 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 04:17:10.558358 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 04:17:10.558358 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 04:17:10.558358 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 04:17:10.558358 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 04:17:10.558358 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 04:17:10.558358 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 04:17:10.558358 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 04:17:10.558358 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 04:17:10.558358 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 04:17:10.558358 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 04:17:10.558358 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 04:17:10.558358 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 04:17:10.558358 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 04:17:10.558358 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 04:17:10.558358 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 04:17:10.558358 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 04:17:10.559214 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 04:17:10.559214 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 04:17:10.559214 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 04:17:10.559214 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 04:17:10.559214 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 04:17:10.559214 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 04:17:10.559214 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 04:17:10.559214 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 04:17:10.559214 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 04:17:10.559214 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 04:17:10.559214 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 06 04:17:10.559214 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 04:17:10.559852 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244251bb-4afc-49db-af03-55c1b9685b42 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:17:10.559922 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 186/350] 10.222.0.30 () {56 vars in 1099 bytes} [Tue Sep 6 04:17:10 2022] GET /identity/v3/roles/member => generated 84 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 04:17:10.564731 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01260949-254f-484d-a074-04e9d0ceef8a None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:10.565017 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01260949-254f-484d-a074-04e9d0ceef8a None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:10.565292 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01260949-254f-484d-a074-04e9d0ceef8a None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:10.567536 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 187/351] 10.222.0.30 () {54 vars in 927 bytes} [Tue Sep 6 04:17:10 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 04:17:10.568840 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01260949-254f-484d-a074-04e9d0ceef8a None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:10.582103 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 165/352] 10.222.0.30 () {56 vars in 1111 bytes} [Tue Sep 6 04:17:10 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 426 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 04:17:10.583864 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e11885d-ce9c-4808-8972-b9bab46c7660 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:10.584151 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e11885d-ce9c-4808-8972-b9bab46c7660 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:10.584431 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e11885d-ce9c-4808-8972-b9bab46c7660 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:10.626412 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-1e11885d-ce9c-4808-8972-b9bab46c7660 None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:17:10.627820 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-1e11885d-ce9c-4808-8972-b9bab46c7660 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:17:10.729134 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01260949-254f-484d-a074-04e9d0ceef8a None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:10.745657 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:17:10.752910 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 166/353] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Sep 6 04:17:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 04:17:10.756954 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e11885d-ce9c-4808-8972-b9bab46c7660 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:10.791440 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:17:10.796227 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493e13af-939a-4e97-b531-73c4ff4cab20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:10.797107 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493e13af-939a-4e97-b531-73c4ff4cab20 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:10.797619 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493e13af-939a-4e97-b531-73c4ff4cab20 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:10.798235 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493e13af-939a-4e97-b531-73c4ff4cab20 None admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:10.799720 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-493e13af-939a-4e97-b531-73c4ff4cab20 None admin] RBAC: Adding query filter params (name=member) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:17:10.800301 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-493e13af-939a-4e97-b531-73c4ff4cab20 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:17:10.802695 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-493e13af-939a-4e97-b531-73c4ff4cab20 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:17:10.820315 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 188/354] 10.222.0.30 () {56 vars in 1101 bytes} [Tue Sep 6 04:17:10 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 04:17:10.825717 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493e13af-939a-4e97-b531-73c4ff4cab20 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:10.976265 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493e13af-939a-4e97-b531-73c4ff4cab20 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:10.980185 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e11885d-ce9c-4808-8972-b9bab46c7660 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:11.019735 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5067aa4f-4ebf-48ce-b2ba-82fb35750221 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:11.021290 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5067aa4f-4ebf-48ce-b2ba-82fb35750221 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:11.021825 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5067aa4f-4ebf-48ce-b2ba-82fb35750221 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:11.022274 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5067aa4f-4ebf-48ce-b2ba-82fb35750221 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:11.074348 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:17:11.082022 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce437ef-7e66-4b3e-b6b3-bdffdd96c7a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:11.082661 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce437ef-7e66-4b3e-b6b3-bdffdd96c7a9 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:11.086998 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce437ef-7e66-4b3e-b6b3-bdffdd96c7a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:11.086998 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce437ef-7e66-4b3e-b6b3-bdffdd96c7a9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:11.109464 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5067aa4f-4ebf-48ce-b2ba-82fb35750221 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:17:11.111136 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5067aa4f-4ebf-48ce-b2ba-82fb35750221 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:17:11.220958 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce437ef-7e66-4b3e-b6b3-bdffdd96c7a9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:17:11.223182 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce437ef-7e66-4b3e-b6b3-bdffdd96c7a9 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:17:11.252807 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:17:11.255402 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 189/355] 10.222.0.30 () {58 vars in 1282 bytes} [Tue Sep 6 04:17:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 430 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 06 04:17:11.256199 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5067aa4f-4ebf-48ce-b2ba-82fb35750221 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:11.360611 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:17:11.362802 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 167/356] 10.222.0.30 () {58 vars in 1282 bytes} [Tue Sep 6 04:17:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 609 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 06 04:17:11.363493 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce437ef-7e66-4b3e-b6b3-bdffdd96c7a9 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:11.401007 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5067aa4f-4ebf-48ce-b2ba-82fb35750221 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:11.452697 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e35c82-2c47-41cb-adda-ed350562f0b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:11.453396 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e35c82-2c47-41cb-adda-ed350562f0b5 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:11.453754 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e35c82-2c47-41cb-adda-ed350562f0b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:11.453945 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e35c82-2c47-41cb-adda-ed350562f0b5 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:11.470428 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e35c82-2c47-41cb-adda-ed350562f0b5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 04:17:11.476492 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-f9e35c82-2c47-41cb-adda-ed350562f0b5 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 06 04:17:11.476492 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 04:17:11.476492 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 04:17:11.476492 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 04:17:11.476492 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 04:17:11.476492 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 04:17:11.476492 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 04:17:11.476492 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 04:17:11.476492 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 04:17:11.476492 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 04:17:11.476492 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 04:17:11.476492 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 04:17:11.476492 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 04:17:11.476492 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 04:17:11.476492 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 04:17:11.476492 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 04:17:11.477631 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 04:17:11.477631 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 04:17:11.477631 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 04:17:11.477631 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 04:17:11.477631 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 04:17:11.477631 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 04:17:11.477631 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 04:17:11.477631 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 04:17:11.477631 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 04:17:11.477631 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 04:17:11.477631 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 04:17:11.477631 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 04:17:11.477631 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 04:17:11.477631 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 04:17:11.477631 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 04:17:11.477631 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 04:17:11.480536 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 04:17:11.480536 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 04:17:11.480536 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 04:17:11.480536 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 04:17:11.480536 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 04:17:11.480536 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 04:17:11.480536 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 04:17:11.480536 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 04:17:11.480536 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 04:17:11.480536 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 04:17:11.480536 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 06 04:17:11.480536 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 04:17:11.480536 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 190/357] 10.222.0.30 () {56 vars in 1114 bytes} [Tue Sep 6 04:17:11 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 04:17:11.480536 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eff31b7-f6ac-492d-956c-18d88b23f03b None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:11.481248 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eff31b7-f6ac-492d-956c-18d88b23f03b None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:11.481248 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eff31b7-f6ac-492d-956c-18d88b23f03b None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:11.482041 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 191/358] 10.222.0.30 () {54 vars in 927 bytes} [Tue Sep 6 04:17:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 04:17:11.482551 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eff31b7-f6ac-492d-956c-18d88b23f03b None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:11.501161 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce437ef-7e66-4b3e-b6b3-bdffdd96c7a9 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:11.531538 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:17:11.553019 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6dc43f-c1d6-4cb2-a50d-87a675afce4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:11.553702 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6dc43f-c1d6-4cb2-a50d-87a675afce4b None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:11.553985 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6dc43f-c1d6-4cb2-a50d-87a675afce4b None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:11.554392 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6dc43f-c1d6-4cb2-a50d-87a675afce4b None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:11.571720 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6dc43f-c1d6-4cb2-a50d-87a675afce4b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 04:17:11.582366 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-af6dc43f-c1d6-4cb2-a50d-87a675afce4b None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 06 04:17:11.582366 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 04:17:11.582366 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 04:17:11.582366 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 04:17:11.582366 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 04:17:11.582366 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 04:17:11.582366 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 04:17:11.582366 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 04:17:11.582366 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 04:17:11.582366 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 04:17:11.582366 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 04:17:11.582366 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 04:17:11.582366 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 04:17:11.582366 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 04:17:11.582366 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 04:17:11.582366 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 04:17:11.583986 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 04:17:11.583986 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 04:17:11.583986 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 04:17:11.583986 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 04:17:11.583986 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 04:17:11.583986 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 04:17:11.583986 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 04:17:11.583986 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 04:17:11.583986 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 04:17:11.583986 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 04:17:11.583986 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 04:17:11.583986 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 04:17:11.583986 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 04:17:11.583986 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 04:17:11.583986 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 04:17:11.583986 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 04:17:11.584962 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 04:17:11.584962 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 04:17:11.584962 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 04:17:11.584962 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 04:17:11.584962 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 04:17:11.584962 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 04:17:11.584962 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 04:17:11.584962 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 04:17:11.584962 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 04:17:11.584962 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 04:17:11.584962 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 06 04:17:11.584962 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 04:17:11.584962 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 168/359] 10.222.0.30 () {56 vars in 1099 bytes} [Tue Sep 6 04:17:11 2022] GET /identity/v3/roles/reader => generated 84 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 04:17:11.585837 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6dc43f-c1d6-4cb2-a50d-87a675afce4b None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:11.634464 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eff31b7-f6ac-492d-956c-18d88b23f03b None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:11.675717 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:17:11.685600 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4795a7a-8fff-4883-b113-c1d4f78ed3c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:11.689224 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4795a7a-8fff-4883-b113-c1d4f78ed3c8 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:11.689224 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4795a7a-8fff-4883-b113-c1d4f78ed3c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:11.689224 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4795a7a-8fff-4883-b113-c1d4f78ed3c8 None admin] PATH_INFO: `/v3/projects/b57372dfc633444d9f127657ca57d59c` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:11.689224 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4795a7a-8fff-4883-b113-c1d4f78ed3c8 None admin] RBAC: Authorizing `identity:get_project(project_id=b57372dfc633444d9f127657ca57d59c)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:17:11.690151 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4795a7a-8fff-4883-b113-c1d4f78ed3c8 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:17:11.691795 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 192/360] 10.222.0.30 () {56 vars in 1186 bytes} [Tue Sep 6 04:17:11 2022] GET /identity/v3/projects/b57372dfc633444d9f127657ca57d59c => generated 302 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:17:11.694846 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4795a7a-8fff-4883-b113-c1d4f78ed3c8 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:11.704099 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6dc43f-c1d6-4cb2-a50d-87a675afce4b None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:11.759055 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d844c0-cd62-48cd-b3a3-c09fe93cbb33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:11.759697 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d844c0-cd62-48cd-b3a3-c09fe93cbb33 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:11.759972 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d844c0-cd62-48cd-b3a3-c09fe93cbb33 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:11.760239 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d844c0-cd62-48cd-b3a3-c09fe93cbb33 None admin] PATH_INFO: `/v3/projects/a88c6977d0554a1eab3e30118ede2f46` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:11.762364 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d844c0-cd62-48cd-b3a3-c09fe93cbb33 None admin] RBAC: Authorizing `identity:get_project(project_id=a88c6977d0554a1eab3e30118ede2f46)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:17:11.764091 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d844c0-cd62-48cd-b3a3-c09fe93cbb33 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:17:11.769356 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 169/361] 10.222.0.30 () {56 vars in 1186 bytes} [Tue Sep 6 04:17:11 2022] GET /identity/v3/projects/a88c6977d0554a1eab3e30118ede2f46 => generated 298 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:17:11.769356 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bcbeb3-e36e-44c4-8735-be14f1e4384a None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:11.769356 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bcbeb3-e36e-44c4-8735-be14f1e4384a None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:11.769356 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bcbeb3-e36e-44c4-8735-be14f1e4384a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:11.788161 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-65bcbeb3-e36e-44c4-8735-be14f1e4384a None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:17:11.789319 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-65bcbeb3-e36e-44c4-8735-be14f1e4384a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:17:11.824126 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4795a7a-8fff-4883-b113-c1d4f78ed3c8 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:11.843745 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:17:11.862735 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26c63a4-c584-4fbe-a314-72f856c96978 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:11.864545 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26c63a4-c584-4fbe-a314-72f856c96978 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:11.864545 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26c63a4-c584-4fbe-a314-72f856c96978 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:11.864545 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26c63a4-c584-4fbe-a314-72f856c96978 None admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:11.865058 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26c63a4-c584-4fbe-a314-72f856c96978 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:17:11.866137 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26c63a4-c584-4fbe-a314-72f856c96978 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:17:11.868365 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26c63a4-c584-4fbe-a314-72f856c96978 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:17:11.875933 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 193/362] 10.222.0.30 () {56 vars in 1111 bytes} [Tue Sep 6 04:17:11 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:17:11.876816 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26c63a4-c584-4fbe-a314-72f856c96978 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:11.878593 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 170/363] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Sep 6 04:17:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 04:17:11.879972 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65bcbeb3-e36e-44c4-8735-be14f1e4384a None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:11.950683 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:17:11.992633 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26c63a4-c584-4fbe-a314-72f856c96978 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:11.999681 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65bcbeb3-e36e-44c4-8735-be14f1e4384a None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:12.029967 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c6ea6f-d866-4cb5-a5c5-bcbf3de97990 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:12.030932 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c6ea6f-d866-4cb5-a5c5-bcbf3de97990 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:12.031451 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c6ea6f-d866-4cb5-a5c5-bcbf3de97990 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:12.031924 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c6ea6f-d866-4cb5-a5c5-bcbf3de97990 None admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:12.032794 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c6ea6f-d866-4cb5-a5c5-bcbf3de97990 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:17:12.033335 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c6ea6f-d866-4cb5-a5c5-bcbf3de97990 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:17:12.034921 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c6ea6f-d866-4cb5-a5c5-bcbf3de97990 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:17:12.038859 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d270435c-158e-4667-a277-d62fc48096b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:12.039447 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d270435c-158e-4667-a277-d62fc48096b9 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:12.039724 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d270435c-158e-4667-a277-d62fc48096b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:12.039995 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d270435c-158e-4667-a277-d62fc48096b9 None admin] PATH_INFO: `/v3/groups/60fbab6f26044826b072daa5861abbb4` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:12.041115 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d270435c-158e-4667-a277-d62fc48096b9 None admin] RBAC: Authorizing `identity:get_group(group_id=60fbab6f26044826b072daa5861abbb4)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:17:12.042751 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d270435c-158e-4667-a277-d62fc48096b9 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:17:12.044392 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 171/364] 10.222.0.30 () {56 vars in 1180 bytes} [Tue Sep 6 04:17:11 2022] GET /identity/v3/groups/60fbab6f26044826b072daa5861abbb4 => generated 226 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:17:12.045242 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d270435c-158e-4667-a277-d62fc48096b9 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:12.046730 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 194/365] 10.222.0.30 () {56 vars in 1101 bytes} [Tue Sep 6 04:17:11 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:17:12.047763 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c6ea6f-d866-4cb5-a5c5-bcbf3de97990 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:12.100157 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:17:12.156607 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c6ea6f-d866-4cb5-a5c5-bcbf3de97990 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:12.162098 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d270435c-158e-4667-a277-d62fc48096b9 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:12.182808 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:17:12.194996 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c43e8f-690a-4cb5-9818-8beb49962eec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:12.195624 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c43e8f-690a-4cb5-9818-8beb49962eec None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:12.195903 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c43e8f-690a-4cb5-9818-8beb49962eec None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:12.196173 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c43e8f-690a-4cb5-9818-8beb49962eec None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:12.202716 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9ce748-a2e7-48b3-932e-d245034085de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:12.203359 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9ce748-a2e7-48b3-932e-d245034085de None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:12.203642 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9ce748-a2e7-48b3-932e-d245034085de None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:12.203913 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9ce748-a2e7-48b3-932e-d245034085de None admin] PATH_INFO: `/v3/groups/60fbab6f26044826b072daa5861abbb4` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:12.205344 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9ce748-a2e7-48b3-932e-d245034085de None admin] RBAC: Authorizing `identity:get_group(group_id=60fbab6f26044826b072daa5861abbb4)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:17:12.210588 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9ce748-a2e7-48b3-932e-d245034085de None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:17:12.210588 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 172/366] 10.222.0.30 () {56 vars in 1180 bytes} [Tue Sep 6 04:17:12 2022] GET /identity/v3/groups/60fbab6f26044826b072daa5861abbb4 => generated 226 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:17:12.210588 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9ce748-a2e7-48b3-932e-d245034085de None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:12.283922 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c43e8f-690a-4cb5-9818-8beb49962eec None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:17:12.285464 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c43e8f-690a-4cb5-9818-8beb49962eec None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:17:12.332060 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9ce748-a2e7-48b3-932e-d245034085de None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:12.353634 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:17:12.385912 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223369f4-b49c-4532-aae9-18f13181396f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:12.386881 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223369f4-b49c-4532-aae9-18f13181396f None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:12.387161 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223369f4-b49c-4532-aae9-18f13181396f None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:12.387432 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223369f4-b49c-4532-aae9-18f13181396f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:12.388172 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-223369f4-b49c-4532-aae9-18f13181396f None admin] RBAC: Adding query filter params (group.id=60fbab6f26044826b072daa5861abbb4, role.id=29607b2de39f40d68bb6ceef64b8a732, scope.project.id=b57372dfc633444d9f127657ca57d59c) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:17:12.388477 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-223369f4-b49c-4532-aae9-18f13181396f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:17:12.390548 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-223369f4-b49c-4532-aae9-18f13181396f None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:17:12.399406 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 173/367] 10.222.0.30 () {56 vars in 1376 bytes} [Tue Sep 6 04:17:12 2022] GET /identity/v3/role_assignments?group.id=60fbab6f26044826b072daa5861abbb4&scope.project.id=b57372dfc633444d9f127657ca57d59c&role.id=29607b2de39f40d68bb6ceef64b8a732 => generated 260 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:17:12.401462 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223369f4-b49c-4532-aae9-18f13181396f None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:12.420993 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 195/368] 10.222.0.30 () {58 vars in 1282 bytes} [Tue Sep 6 04:17:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 06 04:17:12.422720 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c43e8f-690a-4cb5-9818-8beb49962eec None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:12.464661 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:17:12.522540 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223369f4-b49c-4532-aae9-18f13181396f None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:12.569138 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949792c1-a40b-430a-8b62-033a50532bde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:12.570398 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c43e8f-690a-4cb5-9818-8beb49962eec None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:12.570512 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949792c1-a40b-430a-8b62-033a50532bde None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:12.570512 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949792c1-a40b-430a-8b62-033a50532bde None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:12.570512 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949792c1-a40b-430a-8b62-033a50532bde None admin] PATH_INFO: `/v3/users/53431b281da3464cbd805732b370f432` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:12.571744 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949792c1-a40b-430a-8b62-033a50532bde None admin] RBAC: Authorizing `identity:get_user(user_id=53431b281da3464cbd805732b370f432)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:17:12.574701 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949792c1-a40b-430a-8b62-033a50532bde None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:17:12.581330 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 174/369] 10.222.0.30 () {56 vars in 1177 bytes} [Tue Sep 6 04:17:12 2022] GET /identity/v3/users/53431b281da3464cbd805732b370f432 => generated 293 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:17:12.581330 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949792c1-a40b-430a-8b62-033a50532bde None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:12.621611 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26d944c-3843-4bbb-8cb6-930f11861ec1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:12.622175 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26d944c-3843-4bbb-8cb6-930f11861ec1 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:12.622462 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26d944c-3843-4bbb-8cb6-930f11861ec1 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:12.622737 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26d944c-3843-4bbb-8cb6-930f11861ec1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:12.641219 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26d944c-3843-4bbb-8cb6-930f11861ec1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 04:17:12.641900 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:17:12.649075 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-d26d944c-3843-4bbb-8cb6-930f11861ec1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 04:17:12.649075 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 04:17:12.649075 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 04:17:12.649075 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 04:17:12.649075 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 04:17:12.649075 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 04:17:12.649075 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 04:17:12.649075 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 04:17:12.649075 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 04:17:12.649075 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 04:17:12.649075 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 04:17:12.649075 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 04:17:12.649075 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 04:17:12.649075 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 04:17:12.649075 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 04:17:12.649075 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 04:17:12.650003 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 04:17:12.650003 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 04:17:12.650003 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 04:17:12.650003 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 04:17:12.650003 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 04:17:12.650003 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 04:17:12.650003 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 04:17:12.650003 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 04:17:12.650003 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 04:17:12.650003 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 04:17:12.650003 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 04:17:12.650003 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 04:17:12.650003 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 04:17:12.650003 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 04:17:12.650003 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 04:17:12.650003 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 04:17:12.650855 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 04:17:12.650855 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 04:17:12.650855 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 04:17:12.650855 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 04:17:12.650855 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 04:17:12.650855 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 04:17:12.650855 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 04:17:12.650855 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 04:17:12.650855 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 04:17:12.650855 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 04:17:12.650855 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 04:17:12.650855 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 04:17:12.650855 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 196/370] 10.222.0.30 () {56 vars in 1096 bytes} [Tue Sep 6 04:17:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 04:17:12.650855 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26d944c-3843-4bbb-8cb6-930f11861ec1 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:12.751608 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949792c1-a40b-430a-8b62-033a50532bde None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:12.799260 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1596ef2-8780-4083-a5a9-5f871772347c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:12.799889 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1596ef2-8780-4083-a5a9-5f871772347c None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:12.800177 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1596ef2-8780-4083-a5a9-5f871772347c None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:12.800449 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1596ef2-8780-4083-a5a9-5f871772347c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:12.802161 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1596ef2-8780-4083-a5a9-5f871772347c None admin] RBAC: Adding query filter params (group.id=60fbab6f26044826b072daa5861abbb4, role.id=77197446dd2541b9a4fab9a654c08fc9, scope.project.id=a88c6977d0554a1eab3e30118ede2f46) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:17:12.802161 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1596ef2-8780-4083-a5a9-5f871772347c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:17:12.803460 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1596ef2-8780-4083-a5a9-5f871772347c None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:17:12.809295 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:17:12.813240 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 175/371] 10.222.0.30 () {56 vars in 1376 bytes} [Tue Sep 6 04:17:12 2022] GET /identity/v3/role_assignments?group.id=60fbab6f26044826b072daa5861abbb4&scope.project.id=a88c6977d0554a1eab3e30118ede2f46&role.id=77197446dd2541b9a4fab9a654c08fc9 => generated 260 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:17:12.813240 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09721b1e-3b4c-405c-9436-9886088cb72f None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:12.813240 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09721b1e-3b4c-405c-9436-9886088cb72f None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:12.813240 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09721b1e-3b4c-405c-9436-9886088cb72f None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:12.822992 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 176/372] 10.222.0.30 () {54 vars in 927 bytes} [Tue Sep 6 04:17:12 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 04:17:12.823746 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09721b1e-3b4c-405c-9436-9886088cb72f None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:12.835926 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26d944c-3843-4bbb-8cb6-930f11861ec1 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:12.920606 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3077e735-7936-400c-8412-ee3461f23783 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:12.921188 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3077e735-7936-400c-8412-ee3461f23783 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:12.921515 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3077e735-7936-400c-8412-ee3461f23783 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:12.921771 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3077e735-7936-400c-8412-ee3461f23783 None admin] PATH_INFO: `/v3/projects/b57372dfc633444d9f127657ca57d59c` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:12.922950 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3077e735-7936-400c-8412-ee3461f23783 None admin] RBAC: Authorizing `identity:get_project(project_id=b57372dfc633444d9f127657ca57d59c)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:17:12.924432 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3077e735-7936-400c-8412-ee3461f23783 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:17:12.927654 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 197/373] 10.222.0.30 () {56 vars in 1186 bytes} [Tue Sep 6 04:17:12 2022] GET /identity/v3/projects/b57372dfc633444d9f127657ca57d59c => generated 302 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:17:12.929282 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3077e735-7936-400c-8412-ee3461f23783 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:13.011838 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09721b1e-3b4c-405c-9436-9886088cb72f None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:13.034750 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:17:13.069736 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3077e735-7936-400c-8412-ee3461f23783 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:13.096255 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4710827d-bde4-47ff-aaa8-dc14e9f0db56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:13.096880 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4710827d-bde4-47ff-aaa8-dc14e9f0db56 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:13.097159 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4710827d-bde4-47ff-aaa8-dc14e9f0db56 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:13.097469 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4710827d-bde4-47ff-aaa8-dc14e9f0db56 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:13.099129 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4710827d-bde4-47ff-aaa8-dc14e9f0db56 None admin] RBAC: Adding query filter params (role.id=3c011524d1df425b902b8b32c1b485c3, user.id=53431b281da3464cbd805732b370f432, scope.system=all) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:17:13.099129 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4710827d-bde4-47ff-aaa8-dc14e9f0db56 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:17:13.100474 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4710827d-bde4-47ff-aaa8-dc14e9f0db56 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:17:13.114495 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992c0b08-bc03-49ef-9648-b02eab09f58a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:13.115614 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992c0b08-bc03-49ef-9648-b02eab09f58a None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:13.116091 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992c0b08-bc03-49ef-9648-b02eab09f58a None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:13.116576 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992c0b08-bc03-49ef-9648-b02eab09f58a None admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:13.117474 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992c0b08-bc03-49ef-9648-b02eab09f58a None admin] RBAC: Adding query filter params (name=admin) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:17:13.117976 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992c0b08-bc03-49ef-9648-b02eab09f58a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:17:13.118500 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 177/374] 10.222.0.30 () {56 vars in 1308 bytes} [Tue Sep 6 04:17:12 2022] GET /identity/v3/role_assignments?user.id=53431b281da3464cbd805732b370f432&scope.system=all&role.id=3c011524d1df425b902b8b32c1b485c3 => generated 510 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:17:13.119882 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2096965d-c807-4d8b-8b29-4e7d2eb95a72 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:13.120178 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2096965d-c807-4d8b-8b29-4e7d2eb95a72 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:13.120450 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2096965d-c807-4d8b-8b29-4e7d2eb95a72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:13.120662 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992c0b08-bc03-49ef-9648-b02eab09f58a None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:17:13.133106 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 198/375] 10.222.0.30 () {56 vars in 1099 bytes} [Tue Sep 6 04:17:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:17:13.134425 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992c0b08-bc03-49ef-9648-b02eab09f58a None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:13.145296 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-2096965d-c807-4d8b-8b29-4e7d2eb95a72 None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:17:13.148391 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-2096965d-c807-4d8b-8b29-4e7d2eb95a72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:17:13.156539 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:17:13.236313 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 178/376] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Sep 6 04:17:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 04:17:13.236990 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2096965d-c807-4d8b-8b29-4e7d2eb95a72 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:13.264909 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992c0b08-bc03-49ef-9648-b02eab09f58a None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:13.306231 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e09145-5833-484a-9bb8-50615aa8be13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:13.307303 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e09145-5833-484a-9bb8-50615aa8be13 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:13.307782 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e09145-5833-484a-9bb8-50615aa8be13 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:13.308262 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e09145-5833-484a-9bb8-50615aa8be13 None admin] PATH_INFO: `/v3/groups/60fbab6f26044826b072daa5861abbb4` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:13.309832 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e09145-5833-484a-9bb8-50615aa8be13 None admin] RBAC: Authorizing `identity:get_group(group_id=60fbab6f26044826b072daa5861abbb4)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:17:13.311544 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e09145-5833-484a-9bb8-50615aa8be13 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:17:13.313816 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 199/377] 10.222.0.30 () {56 vars in 1180 bytes} [Tue Sep 6 04:17:13 2022] GET /identity/v3/groups/60fbab6f26044826b072daa5861abbb4 => generated 226 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:17:13.320255 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e09145-5833-484a-9bb8-50615aa8be13 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:13.426042 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:17:13.449854 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e09145-5833-484a-9bb8-50615aa8be13 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:13.460357 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:17:13.471417 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2096965d-c807-4d8b-8b29-4e7d2eb95a72 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:13.498528 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e5271c-3712-44cf-9012-3c9388dfbe58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:13.498528 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e5271c-3712-44cf-9012-3c9388dfbe58 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:13.498528 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e5271c-3712-44cf-9012-3c9388dfbe58 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:13.498528 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e5271c-3712-44cf-9012-3c9388dfbe58 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:13.498951 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e5271c-3712-44cf-9012-3c9388dfbe58 None admin] RBAC: Adding query filter params (group.id=60fbab6f26044826b072daa5861abbb4, role.id=77197446dd2541b9a4fab9a654c08fc9, scope.project.id=b57372dfc633444d9f127657ca57d59c) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:17:13.498951 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e5271c-3712-44cf-9012-3c9388dfbe58 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:17:13.500254 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e5271c-3712-44cf-9012-3c9388dfbe58 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:17:13.508675 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 200/378] 10.222.0.30 () {56 vars in 1376 bytes} [Tue Sep 6 04:17:13 2022] GET /identity/v3/role_assignments?group.id=60fbab6f26044826b072daa5861abbb4&scope.project.id=b57372dfc633444d9f127657ca57d59c&role.id=77197446dd2541b9a4fab9a654c08fc9 => generated 260 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 04:17:13.535321 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a6ef0a-731b-4cc1-8a44-63482a696e25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:13.536125 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a6ef0a-731b-4cc1-8a44-63482a696e25 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:13.536405 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a6ef0a-731b-4cc1-8a44-63482a696e25 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:13.536677 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a6ef0a-731b-4cc1-8a44-63482a696e25 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:13.629923 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a6ef0a-731b-4cc1-8a44-63482a696e25 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:17:13.631395 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a6ef0a-731b-4cc1-8a44-63482a696e25 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:17:13.714935 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e5271c-3712-44cf-9012-3c9388dfbe58 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:13.755994 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:17:13.766223 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 179/379] 10.222.0.30 () {58 vars in 1282 bytes} [Tue Sep 6 04:17:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 530 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 06 04:17:13.772081 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a6ef0a-731b-4cc1-8a44-63482a696e25 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:13.866252 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e5271c-3712-44cf-9012-3c9388dfbe58 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:13.906883 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a6ef0a-731b-4cc1-8a44-63482a696e25 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:13.913745 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5821ef-ad9c-4266-b52f-6a416981f314 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:13.914619 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5821ef-ad9c-4266-b52f-6a416981f314 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:13.915653 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5821ef-ad9c-4266-b52f-6a416981f314 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:13.916107 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5821ef-ad9c-4266-b52f-6a416981f314 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:13.931565 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5821ef-ad9c-4266-b52f-6a416981f314 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 04:17:13.937866 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-0d5821ef-ad9c-4266-b52f-6a416981f314 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 06 04:17:13.937866 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 04:17:13.937866 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 04:17:13.937866 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 04:17:13.937866 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 04:17:13.937866 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 04:17:13.937866 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 04:17:13.937866 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 04:17:13.937866 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 04:17:13.937866 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 04:17:13.937866 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 04:17:13.937866 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 04:17:13.937866 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 04:17:13.937866 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 04:17:13.937866 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 04:17:13.937866 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 04:17:13.938813 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 04:17:13.938813 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 04:17:13.938813 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 04:17:13.938813 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 04:17:13.938813 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 04:17:13.938813 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 04:17:13.938813 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 04:17:13.938813 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 04:17:13.938813 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 04:17:13.938813 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 04:17:13.938813 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 04:17:13.938813 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 04:17:13.938813 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 04:17:13.938813 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 04:17:13.938813 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 04:17:13.938813 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 04:17:13.939915 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 04:17:13.939915 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 04:17:13.939915 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 04:17:13.939915 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 04:17:13.939915 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 04:17:13.939915 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 04:17:13.939915 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 04:17:13.939915 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 04:17:13.939915 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 04:17:13.939915 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 04:17:13.939915 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 06 04:17:13.939915 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 04:17:13.939915 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 201/380] 10.222.0.30 () {56 vars in 1099 bytes} [Tue Sep 6 04:17:13 2022] GET /identity/v3/roles/member => generated 84 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 04:17:13.946983 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5821ef-ad9c-4266-b52f-6a416981f314 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:13.954409 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f0477a-aa50-4778-a81b-eea25051c488 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:13.955281 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f0477a-aa50-4778-a81b-eea25051c488 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:13.955737 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f0477a-aa50-4778-a81b-eea25051c488 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:13.956215 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f0477a-aa50-4778-a81b-eea25051c488 None admin] PATH_INFO: `/v3/projects/8b3067b612904279b76189015d91b056` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:13.958543 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:17:13.959601 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f0477a-aa50-4778-a81b-eea25051c488 None admin] RBAC: Authorizing `identity:get_project(project_id=8b3067b612904279b76189015d91b056)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:17:13.962370 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f0477a-aa50-4778-a81b-eea25051c488 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:17:13.964604 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 180/381] 10.222.0.30 () {56 vars in 1186 bytes} [Tue Sep 6 04:17:13 2022] GET /identity/v3/projects/8b3067b612904279b76189015d91b056 => generated 344 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 04:17:13.969706 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f0477a-aa50-4778-a81b-eea25051c488 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:14.115533 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:17:14.142279 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5821ef-ad9c-4266-b52f-6a416981f314 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:14.156883 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f0477a-aa50-4778-a81b-eea25051c488 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:14.199568 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dceca8fb-126e-4d2d-ae31-5df0ad36a16d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:14.200211 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dceca8fb-126e-4d2d-ae31-5df0ad36a16d None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:14.200547 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dceca8fb-126e-4d2d-ae31-5df0ad36a16d None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:14.200822 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dceca8fb-126e-4d2d-ae31-5df0ad36a16d None admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:14.201541 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dceca8fb-126e-4d2d-ae31-5df0ad36a16d None admin] RBAC: Adding query filter params (name=member) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:17:14.201844 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dceca8fb-126e-4d2d-ae31-5df0ad36a16d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:17:14.203289 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dceca8fb-126e-4d2d-ae31-5df0ad36a16d None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:17:14.224665 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 202/382] 10.222.0.30 () {56 vars in 1101 bytes} [Tue Sep 6 04:17:13 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:17:14.230552 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dceca8fb-126e-4d2d-ae31-5df0ad36a16d None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:14.232753 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58853f13-4f74-4047-8740-8ca0cf05f86b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:14.233433 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58853f13-4f74-4047-8740-8ca0cf05f86b None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:14.233714 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58853f13-4f74-4047-8740-8ca0cf05f86b None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:14.233984 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58853f13-4f74-4047-8740-8ca0cf05f86b None admin] PATH_INFO: `/v3/groups/00b3acdc6a4443d6904694fcb386f700` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:14.248614 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58853f13-4f74-4047-8740-8ca0cf05f86b None admin] RBAC: Authorizing `identity:get_group(group_id=00b3acdc6a4443d6904694fcb386f700)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:17:14.250161 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58853f13-4f74-4047-8740-8ca0cf05f86b None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:17:14.251881 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 181/383] 10.222.0.30 () {56 vars in 1180 bytes} [Tue Sep 6 04:17:13 2022] GET /identity/v3/groups/00b3acdc6a4443d6904694fcb386f700 => generated 229 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 04:17:14.258464 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58853f13-4f74-4047-8740-8ca0cf05f86b None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:14.385034 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:17:14.393262 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58853f13-4f74-4047-8740-8ca0cf05f86b None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:14.424452 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:17:14.432995 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7344ffd-a9af-48be-8d63-27dab1c181dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:14.433698 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7344ffd-a9af-48be-8d63-27dab1c181dd None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:14.433976 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7344ffd-a9af-48be-8d63-27dab1c181dd None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:14.434264 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7344ffd-a9af-48be-8d63-27dab1c181dd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:14.434986 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7344ffd-a9af-48be-8d63-27dab1c181dd None admin] RBAC: Adding query filter params (group.id=00b3acdc6a4443d6904694fcb386f700, role.id=ea4c814713b149108e488de855b025e4, scope.project.id=8b3067b612904279b76189015d91b056) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:17:14.435906 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7344ffd-a9af-48be-8d63-27dab1c181dd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:17:14.436706 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7344ffd-a9af-48be-8d63-27dab1c181dd None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:17:14.444157 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 182/384] 10.222.0.30 () {56 vars in 1376 bytes} [Tue Sep 6 04:17:14 2022] GET /identity/v3/role_assignments?group.id=00b3acdc6a4443d6904694fcb386f700&scope.project.id=8b3067b612904279b76189015d91b056&role.id=ea4c814713b149108e488de855b025e4 => generated 260 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:17:14.465781 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dceca8fb-126e-4d2d-ae31-5df0ad36a16d None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:14.530159 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f57934-195b-4ac9-8ef5-315927c64063 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:14.530838 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f57934-195b-4ac9-8ef5-315927c64063 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:14.531125 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f57934-195b-4ac9-8ef5-315927c64063 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:14.531412 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f57934-195b-4ac9-8ef5-315927c64063 None admin] PATH_INFO: `/v3/groups/60fbab6f26044826b072daa5861abbb4` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:14.532716 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f57934-195b-4ac9-8ef5-315927c64063 None admin] RBAC: Authorizing `identity:get_group(group_id=60fbab6f26044826b072daa5861abbb4)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:17:14.534244 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f57934-195b-4ac9-8ef5-315927c64063 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:17:14.536723 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 203/385] 10.222.0.30 () {56 vars in 1180 bytes} [Tue Sep 6 04:17:14 2022] GET /identity/v3/groups/60fbab6f26044826b072daa5861abbb4 => generated 226 bytes in 306 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:17:14.542546 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7344ffd-a9af-48be-8d63-27dab1c181dd None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:14.712523 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7344ffd-a9af-48be-8d63-27dab1c181dd None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:14.790549 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f38274e-50a8-4364-87c8-9130dca646c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:14.792092 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f38274e-50a8-4364-87c8-9130dca646c4 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:14.792832 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f38274e-50a8-4364-87c8-9130dca646c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:14.793642 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f38274e-50a8-4364-87c8-9130dca646c4 None admin] PATH_INFO: `/v3/projects/a88c6977d0554a1eab3e30118ede2f46` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:14.795814 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f38274e-50a8-4364-87c8-9130dca646c4 None admin] RBAC: Authorizing `identity:get_project(project_id=a88c6977d0554a1eab3e30118ede2f46)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:17:14.798703 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f38274e-50a8-4364-87c8-9130dca646c4 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:17:14.806620 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 183/386] 10.222.0.30 () {56 vars in 1186 bytes} [Tue Sep 6 04:17:14 2022] GET /identity/v3/projects/a88c6977d0554a1eab3e30118ede2f46 => generated 298 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:17:14.811860 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f57934-195b-4ac9-8ef5-315927c64063 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:14.940549 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f57934-195b-4ac9-8ef5-315927c64063 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:14.979471 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd82be0-aca0-4d96-83c0-9d26a77aad49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:14.980114 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd82be0-aca0-4d96-83c0-9d26a77aad49 None admin] REQUEST_METHOD: `PUT` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:14.980399 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd82be0-aca0-4d96-83c0-9d26a77aad49 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:14.980680 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd82be0-aca0-4d96-83c0-9d26a77aad49 None admin] PATH_INFO: `/v3/projects/a88c6977d0554a1eab3e30118ede2f46/groups/60fbab6f26044826b072daa5861abbb4/roles/29607b2de39f40d68bb6ceef64b8a732` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:14.982768 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd82be0-aca0-4d96-83c0-9d26a77aad49 None admin] RBAC: Authorizing `identity:create_grant(project_id=a88c6977d0554a1eab3e30118ede2f46, group_id=60fbab6f26044826b072daa5861abbb4, role_id=29607b2de39f40d68bb6ceef64b8a732)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:17:14.984212 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd82be0-aca0-4d96-83c0-9d26a77aad49 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:17:14.988786 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:17:15.077519 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 204/387] 10.222.0.30 () {58 vars in 1442 bytes} [Tue Sep 6 04:17:14 2022] PUT /identity/v3/projects/a88c6977d0554a1eab3e30118ede2f46/groups/60fbab6f26044826b072daa5861abbb4/roles/29607b2de39f40d68bb6ceef64b8a732 => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 04:17:15.317481 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efdb1d2-d267-4b96-944a-3225233b7b7a None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:15.317481 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efdb1d2-d267-4b96-944a-3225233b7b7a None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:15.317481 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efdb1d2-d267-4b96-944a-3225233b7b7a None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:15.318732 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 184/388] 10.222.0.30 () {54 vars in 927 bytes} [Tue Sep 6 04:17:15 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 04:17:15.324599 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426c0eb7-83fd-4a02-8978-683f1fa3b427 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:15.324905 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426c0eb7-83fd-4a02-8978-683f1fa3b427 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:15.325168 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426c0eb7-83fd-4a02-8978-683f1fa3b427 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:15.353759 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-426c0eb7-83fd-4a02-8978-683f1fa3b427 None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:17:15.354968 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-426c0eb7-83fd-4a02-8978-683f1fa3b427 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:17:15.455483 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 205/389] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Sep 6 04:17:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 06 04:17:15.521991 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef232ad-397e-4cda-8fa2-ef01951c0f1c None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:15.521991 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef232ad-397e-4cda-8fa2-ef01951c0f1c None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:15.521991 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef232ad-397e-4cda-8fa2-ef01951c0f1c None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:15.521991 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 185/390] 10.222.0.30 () {54 vars in 927 bytes} [Tue Sep 6 04:17:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 04:17:15.527240 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64498d3a-7d52-449c-b116-336fb6a19c56 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:15.527703 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64498d3a-7d52-449c-b116-336fb6a19c56 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:15.527777 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64498d3a-7d52-449c-b116-336fb6a19c56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:15.554314 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-64498d3a-7d52-449c-b116-336fb6a19c56 None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:17:15.555523 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-64498d3a-7d52-449c-b116-336fb6a19c56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:17:15.639916 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 206/391] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Sep 6 04:17:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 04:17:15.809608 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef232ad-397e-4cda-8fa2-ef01951c0f1c None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:15.899815 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:17:15.950307 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef232ad-397e-4cda-8fa2-ef01951c0f1c None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:15.991015 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64498d3a-7d52-449c-b116-336fb6a19c56 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:16.003089 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0262aa61-9134-4dfb-98d7-67d8dae1ecb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:16.003731 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0262aa61-9134-4dfb-98d7-67d8dae1ecb7 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:16.004008 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0262aa61-9134-4dfb-98d7-67d8dae1ecb7 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:16.004278 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0262aa61-9134-4dfb-98d7-67d8dae1ecb7 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:16.018418 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0262aa61-9134-4dfb-98d7-67d8dae1ecb7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 04:17:16.023800 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-0262aa61-9134-4dfb-98d7-67d8dae1ecb7 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 06 04:17:16.023800 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 04:17:16.023800 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 04:17:16.023800 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 04:17:16.023800 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 04:17:16.023800 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 04:17:16.023800 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 04:17:16.023800 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 04:17:16.023800 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 04:17:16.023800 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 04:17:16.023800 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 04:17:16.023800 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 04:17:16.023800 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 04:17:16.023800 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 04:17:16.023800 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 04:17:16.023800 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 04:17:16.024675 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 04:17:16.024675 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 04:17:16.024675 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 04:17:16.024675 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 04:17:16.024675 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 04:17:16.024675 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 04:17:16.024675 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 04:17:16.024675 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 04:17:16.024675 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 04:17:16.024675 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 04:17:16.024675 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 04:17:16.024675 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 04:17:16.024675 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 04:17:16.024675 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 04:17:16.024675 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 04:17:16.024675 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 04:17:16.025490 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 04:17:16.025490 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 04:17:16.025490 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 04:17:16.025490 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 04:17:16.025490 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 04:17:16.025490 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 04:17:16.025490 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 04:17:16.025490 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 04:17:16.025490 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 04:17:16.025490 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 04:17:16.025490 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 06 04:17:16.025490 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 04:17:16.029410 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 186/392] 10.222.0.30 () {56 vars in 1099 bytes} [Tue Sep 6 04:17:15 2022] GET /identity/v3/roles/member => generated 84 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 04:17:16.030912 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0262aa61-9134-4dfb-98d7-67d8dae1ecb7 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:16.136818 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0262aa61-9134-4dfb-98d7-67d8dae1ecb7 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:16.138660 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:17:16.161172 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:17:16.172913 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5daa2837-c41c-415b-ad4f-8510560e99cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:16.173561 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5daa2837-c41c-415b-ad4f-8510560e99cc None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:16.173843 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5daa2837-c41c-415b-ad4f-8510560e99cc None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:16.174117 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5daa2837-c41c-415b-ad4f-8510560e99cc None admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:16.174893 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5daa2837-c41c-415b-ad4f-8510560e99cc None admin] RBAC: Adding query filter params (name=member) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:17:16.175199 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5daa2837-c41c-415b-ad4f-8510560e99cc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:17:16.176671 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5daa2837-c41c-415b-ad4f-8510560e99cc None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:17:16.190523 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 187/393] 10.222.0.30 () {56 vars in 1101 bytes} [Tue Sep 6 04:17:16 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 04:17:16.192408 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5daa2837-c41c-415b-ad4f-8510560e99cc None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:16.264059 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64498d3a-7d52-449c-b116-336fb6a19c56 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:16.301948 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5daa2837-c41c-415b-ad4f-8510560e99cc None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:16.315540 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55b5dfd-22e8-481b-a7ce-09b3b17566d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:16.316716 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55b5dfd-22e8-481b-a7ce-09b3b17566d7 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:16.317028 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55b5dfd-22e8-481b-a7ce-09b3b17566d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:16.317472 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55b5dfd-22e8-481b-a7ce-09b3b17566d7 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:16.331708 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55b5dfd-22e8-481b-a7ce-09b3b17566d7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 04:17:16.337538 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdca0c6c-1703-40e3-8f26-1246d14c5aef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:16.337838 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-e55b5dfd-22e8-481b-a7ce-09b3b17566d7 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 06 04:17:16.337838 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 04:17:16.337838 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 04:17:16.337838 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 04:17:16.337838 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 04:17:16.337838 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 04:17:16.337838 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 04:17:16.337838 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 04:17:16.337838 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 04:17:16.337838 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 04:17:16.337838 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 04:17:16.337838 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 04:17:16.337838 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 04:17:16.337838 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 04:17:16.337838 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 04:17:16.337838 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 04:17:16.338710 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 04:17:16.338710 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 04:17:16.338710 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 04:17:16.338710 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 04:17:16.338710 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 04:17:16.338710 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 04:17:16.338710 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 04:17:16.338710 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 04:17:16.338710 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 04:17:16.338710 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 04:17:16.338710 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 04:17:16.338710 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 04:17:16.338710 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 04:17:16.338710 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 04:17:16.338710 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 04:17:16.338710 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 04:17:16.339522 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 04:17:16.339522 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 04:17:16.339522 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 04:17:16.339522 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 04:17:16.339522 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 04:17:16.339522 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 04:17:16.339522 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 04:17:16.339522 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 04:17:16.339522 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 04:17:16.339522 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 04:17:16.339522 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 06 04:17:16.339522 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 04:17:16.340113 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdca0c6c-1703-40e3-8f26-1246d14c5aef None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:16.340113 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdca0c6c-1703-40e3-8f26-1246d14c5aef None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:16.340113 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdca0c6c-1703-40e3-8f26-1246d14c5aef None admin] PATH_INFO: `/v3/groups/60fbab6f26044826b072daa5861abbb4` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:16.340277 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 207/394] 10.222.0.30 () {56 vars in 1114 bytes} [Tue Sep 6 04:17:15 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 349 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 04:17:16.340277 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87532ae3-0662-449d-92b5-a6524d514cbd None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:16.341089 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87532ae3-0662-449d-92b5-a6524d514cbd None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:16.341089 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87532ae3-0662-449d-92b5-a6524d514cbd None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:16.342281 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdca0c6c-1703-40e3-8f26-1246d14c5aef None admin] RBAC: Authorizing `identity:get_group(group_id=60fbab6f26044826b072daa5861abbb4)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:17:16.342390 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 208/395] 10.222.0.30 () {54 vars in 927 bytes} [Tue Sep 6 04:17:16 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 04:17:16.347244 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094bc8bd-87f6-4d46-8c78-34beead40c30 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:16.347456 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdca0c6c-1703-40e3-8f26-1246d14c5aef None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:17:16.347723 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094bc8bd-87f6-4d46-8c78-34beead40c30 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:16.348097 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094bc8bd-87f6-4d46-8c78-34beead40c30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:16.349522 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 188/396] 10.222.0.30 () {56 vars in 1180 bytes} [Tue Sep 6 04:17:16 2022] GET /identity/v3/groups/60fbab6f26044826b072daa5861abbb4 => generated 226 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:17:16.350347 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdca0c6c-1703-40e3-8f26-1246d14c5aef None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:16.378993 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-094bc8bd-87f6-4d46-8c78-34beead40c30 None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:17:16.380968 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-094bc8bd-87f6-4d46-8c78-34beead40c30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:17:16.417530 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:17:16.467547 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdca0c6c-1703-40e3-8f26-1246d14c5aef None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:16.478510 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 209/397] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Sep 6 04:17:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 04:17:16.480183 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094bc8bd-87f6-4d46-8c78-34beead40c30 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:16.534004 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5199568b-9081-45e1-8429-a56dc7e72f5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:16.535021 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5199568b-9081-45e1-8429-a56dc7e72f5e None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:16.536439 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5199568b-9081-45e1-8429-a56dc7e72f5e None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:16.537069 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5199568b-9081-45e1-8429-a56dc7e72f5e None admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:16.538487 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5199568b-9081-45e1-8429-a56dc7e72f5e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:17:16.539128 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5199568b-9081-45e1-8429-a56dc7e72f5e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:17:16.541271 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5199568b-9081-45e1-8429-a56dc7e72f5e None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:17:16.560772 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 189/398] 10.222.0.30 () {56 vars in 1111 bytes} [Tue Sep 6 04:17:16 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:17:16.565868 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5199568b-9081-45e1-8429-a56dc7e72f5e None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:16.578868 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:17:16.669831 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094bc8bd-87f6-4d46-8c78-34beead40c30 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:16.725582 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d67d9f-8997-4c80-97b1-a16ed9ba64d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:16.727850 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d67d9f-8997-4c80-97b1-a16ed9ba64d4 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:16.729642 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d67d9f-8997-4c80-97b1-a16ed9ba64d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:16.730064 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d67d9f-8997-4c80-97b1-a16ed9ba64d4 None admin] PATH_INFO: `/v3/projects/b57372dfc633444d9f127657ca57d59c` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:16.732247 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d67d9f-8997-4c80-97b1-a16ed9ba64d4 None admin] RBAC: Authorizing `identity:get_project(project_id=b57372dfc633444d9f127657ca57d59c)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:17:16.733966 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d67d9f-8997-4c80-97b1-a16ed9ba64d4 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:17:16.738410 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 210/399] 10.222.0.30 () {56 vars in 1186 bytes} [Tue Sep 6 04:17:16 2022] GET /identity/v3/projects/b57372dfc633444d9f127657ca57d59c => generated 302 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:17:16.754978 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d67d9f-8997-4c80-97b1-a16ed9ba64d4 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:16.768691 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5199568b-9081-45e1-8429-a56dc7e72f5e None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:16.787933 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:17:16.808367 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b7b454-88db-425e-801c-207194bebdf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:16.809100 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b7b454-88db-425e-801c-207194bebdf5 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:16.809416 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b7b454-88db-425e-801c-207194bebdf5 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:16.816506 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b7b454-88db-425e-801c-207194bebdf5 None admin] PATH_INFO: `/v3/groups/60fbab6f26044826b072daa5861abbb4` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:16.816506 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b7b454-88db-425e-801c-207194bebdf5 None admin] RBAC: Authorizing `identity:get_group(group_id=60fbab6f26044826b072daa5861abbb4)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:17:16.816506 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b7b454-88db-425e-801c-207194bebdf5 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:17:16.816506 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 190/400] 10.222.0.30 () {56 vars in 1180 bytes} [Tue Sep 6 04:17:16 2022] GET /identity/v3/groups/60fbab6f26044826b072daa5861abbb4 => generated 226 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:17:16.816506 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b7b454-88db-425e-801c-207194bebdf5 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:16.904380 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d67d9f-8997-4c80-97b1-a16ed9ba64d4 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:16.922222 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:17:16.944383 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3b451e-e73d-423a-967f-3aefb767946e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:16.944573 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b7b454-88db-425e-801c-207194bebdf5 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:16.945548 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3b451e-e73d-423a-967f-3aefb767946e None admin] REQUEST_METHOD: `PUT` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:16.946050 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3b451e-e73d-423a-967f-3aefb767946e None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:16.946514 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3b451e-e73d-423a-967f-3aefb767946e None admin] PATH_INFO: `/v3/projects/b57372dfc633444d9f127657ca57d59c/groups/60fbab6f26044826b072daa5861abbb4/roles/29607b2de39f40d68bb6ceef64b8a732` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:16.948837 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b3b451e-e73d-423a-967f-3aefb767946e None admin] RBAC: Authorizing `identity:create_grant(project_id=b57372dfc633444d9f127657ca57d59c, group_id=60fbab6f26044826b072daa5861abbb4, role_id=29607b2de39f40d68bb6ceef64b8a732)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:17:16.950518 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b3b451e-e73d-423a-967f-3aefb767946e None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:17:16.981116 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba45e66-7a51-4542-9842-939c6c064619 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:16.981781 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba45e66-7a51-4542-9842-939c6c064619 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:16.982058 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba45e66-7a51-4542-9842-939c6c064619 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:16.982328 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba45e66-7a51-4542-9842-939c6c064619 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:16.992993 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba45e66-7a51-4542-9842-939c6c064619 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 04:17:16.998311 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-7ba45e66-7a51-4542-9842-939c6c064619 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 06 04:17:16.998311 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 04:17:16.998311 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 04:17:16.998311 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 04:17:16.998311 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 04:17:16.998311 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 04:17:16.998311 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 04:17:16.998311 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 04:17:16.998311 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 04:17:16.998311 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 04:17:16.998311 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 04:17:16.998311 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 04:17:16.998311 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 04:17:16.998311 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 04:17:16.998311 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 04:17:16.998311 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 04:17:16.999203 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 04:17:16.999203 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 04:17:16.999203 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 04:17:16.999203 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 04:17:16.999203 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 04:17:16.999203 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 04:17:16.999203 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 04:17:16.999203 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 04:17:16.999203 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 04:17:16.999203 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 04:17:16.999203 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 04:17:16.999203 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 04:17:16.999203 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 04:17:16.999203 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 04:17:16.999203 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 04:17:16.999203 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 04:17:17.000751 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 04:17:17.000751 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 04:17:17.000751 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 04:17:17.000751 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 04:17:17.000751 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 04:17:17.000751 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 04:17:17.000751 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 04:17:17.000751 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 04:17:17.000751 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 04:17:17.000751 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 04:17:17.000751 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 06 04:17:17.000751 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 04:17:17.000751 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 191/401] 10.222.0.30 () {56 vars in 1114 bytes} [Tue Sep 6 04:17:16 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 04:17:17.000751 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba45e66-7a51-4542-9842-939c6c064619 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:17.044225 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:17:17.059255 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 211/402] 10.222.0.30 () {58 vars in 1442 bytes} [Tue Sep 6 04:17:16 2022] PUT /identity/v3/projects/b57372dfc633444d9f127657ca57d59c/groups/60fbab6f26044826b072daa5861abbb4/roles/29607b2de39f40d68bb6ceef64b8a732 => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 04:17:17.060590 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c380b9ec-2737-44a9-bba6-02bda8b455fe None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:17.060972 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c380b9ec-2737-44a9-bba6-02bda8b455fe None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:17.061330 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c380b9ec-2737-44a9-bba6-02bda8b455fe None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:17.062889 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 212/403] 10.222.0.30 () {54 vars in 927 bytes} [Tue Sep 6 04:17:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 04:17:17.066322 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c380b9ec-2737-44a9-bba6-02bda8b455fe None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:17.120052 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba45e66-7a51-4542-9842-939c6c064619 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:17.186902 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34878155-34da-4304-9c1e-bddbe947a95b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:17.187941 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34878155-34da-4304-9c1e-bddbe947a95b None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:17.188305 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34878155-34da-4304-9c1e-bddbe947a95b None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:17.188722 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34878155-34da-4304-9c1e-bddbe947a95b None admin] PATH_INFO: `/v3/projects/a88c6977d0554a1eab3e30118ede2f46` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:17.190400 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34878155-34da-4304-9c1e-bddbe947a95b None admin] RBAC: Authorizing `identity:get_project(project_id=a88c6977d0554a1eab3e30118ede2f46)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:17:17.193795 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34878155-34da-4304-9c1e-bddbe947a95b None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:17:17.202012 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 192/404] 10.222.0.30 () {56 vars in 1186 bytes} [Tue Sep 6 04:17:16 2022] GET /identity/v3/projects/a88c6977d0554a1eab3e30118ede2f46 => generated 298 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:17:17.204398 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3315c8a6-997d-4c4c-b4ff-49a16b710fc7 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:17.204767 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3315c8a6-997d-4c4c-b4ff-49a16b710fc7 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:17.205122 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3315c8a6-997d-4c4c-b4ff-49a16b710fc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:17.207663 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c380b9ec-2737-44a9-bba6-02bda8b455fe None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:17.229247 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-3315c8a6-997d-4c4c-b4ff-49a16b710fc7 None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:17:17.229576 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-3315c8a6-997d-4c4c-b4ff-49a16b710fc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:17:17.253213 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce967f3-d0f9-4aac-8767-62bebd98d3c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:17.253870 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce967f3-d0f9-4aac-8767-62bebd98d3c4 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:17.254138 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce967f3-d0f9-4aac-8767-62bebd98d3c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:17.254475 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce967f3-d0f9-4aac-8767-62bebd98d3c4 None admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:17.255826 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce967f3-d0f9-4aac-8767-62bebd98d3c4 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:17:17.256126 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce967f3-d0f9-4aac-8767-62bebd98d3c4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:17:17.258190 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce967f3-d0f9-4aac-8767-62bebd98d3c4 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:17:17.267221 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 213/405] 10.222.0.30 () {56 vars in 1111 bytes} [Tue Sep 6 04:17:17 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:17:17.267865 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce967f3-d0f9-4aac-8767-62bebd98d3c4 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:17.309692 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 193/406] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Sep 6 04:17:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 04:17:17.310562 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3315c8a6-997d-4c4c-b4ff-49a16b710fc7 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:17.318684 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:17:17.361259 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:17:17.431377 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3315c8a6-997d-4c4c-b4ff-49a16b710fc7 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:17.474104 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb678e0-1794-4489-9704-4cc8152ae5e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:17.475231 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb678e0-1794-4489-9704-4cc8152ae5e7 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:17.475712 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb678e0-1794-4489-9704-4cc8152ae5e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:17.476183 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb678e0-1794-4489-9704-4cc8152ae5e7 None admin] PATH_INFO: `/v3/groups/60fbab6f26044826b072daa5861abbb4` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:17.477841 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb678e0-1794-4489-9704-4cc8152ae5e7 None admin] RBAC: Authorizing `identity:get_group(group_id=60fbab6f26044826b072daa5861abbb4)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:17:17.479275 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce967f3-d0f9-4aac-8767-62bebd98d3c4 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:17.479770 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb678e0-1794-4489-9704-4cc8152ae5e7 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:17:17.483139 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 194/407] 10.222.0.30 () {56 vars in 1180 bytes} [Tue Sep 6 04:17:17 2022] GET /identity/v3/groups/60fbab6f26044826b072daa5861abbb4 => generated 226 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:17:17.484736 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86a0e14-6f3f-47c2-aaa6-f89cb66b83c0 None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:17.485244 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86a0e14-6f3f-47c2-aaa6-f89cb66b83c0 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:17.485736 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86a0e14-6f3f-47c2-aaa6-f89cb66b83c0 None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:17.487832 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 195/408] 10.222.0.30 () {54 vars in 927 bytes} [Tue Sep 6 04:17:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 04:17:17.489900 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86a0e14-6f3f-47c2-aaa6-f89cb66b83c0 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:17.524318 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f60f38-fbd5-4641-a2b8-4d1afa3d7b1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:17.525042 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f60f38-fbd5-4641-a2b8-4d1afa3d7b1f None admin] REQUEST_METHOD: `PUT` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:17.525424 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f60f38-fbd5-4641-a2b8-4d1afa3d7b1f None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:17.525635 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f60f38-fbd5-4641-a2b8-4d1afa3d7b1f None admin] PATH_INFO: `/v3/projects/a88c6977d0554a1eab3e30118ede2f46/groups/60fbab6f26044826b072daa5861abbb4/roles/77197446dd2541b9a4fab9a654c08fc9` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:17.527889 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f60f38-fbd5-4641-a2b8-4d1afa3d7b1f None admin] RBAC: Authorizing `identity:create_grant(project_id=a88c6977d0554a1eab3e30118ede2f46, group_id=60fbab6f26044826b072daa5861abbb4, role_id=77197446dd2541b9a4fab9a654c08fc9)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:17:17.529385 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f60f38-fbd5-4641-a2b8-4d1afa3d7b1f None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:17:17.614733 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86a0e14-6f3f-47c2-aaa6-f89cb66b83c0 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:17.624763 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:17:17.637873 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 214/409] 10.222.0.30 () {58 vars in 1442 bytes} [Tue Sep 6 04:17:17 2022] PUT /identity/v3/projects/a88c6977d0554a1eab3e30118ede2f46/groups/60fbab6f26044826b072daa5861abbb4/roles/77197446dd2541b9a4fab9a654c08fc9 => generated 0 bytes in 370 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 04:17:17.639139 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deaaf407-940f-4718-a307-7a69863da096 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:17.639425 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deaaf407-940f-4718-a307-7a69863da096 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:17.639690 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deaaf407-940f-4718-a307-7a69863da096 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:17.660103 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-deaaf407-940f-4718-a307-7a69863da096 None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:17:17.661700 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-deaaf407-940f-4718-a307-7a69863da096 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:17:17.667766 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c334b810-69e8-4fce-b4ce-7f84ce7f040c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:17.668889 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c334b810-69e8-4fce-b4ce-7f84ce7f040c None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:17.669413 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c334b810-69e8-4fce-b4ce-7f84ce7f040c None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:17.669883 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c334b810-69e8-4fce-b4ce-7f84ce7f040c None admin] PATH_INFO: `/v3/projects/b57372dfc633444d9f127657ca57d59c` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:17.672401 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c334b810-69e8-4fce-b4ce-7f84ce7f040c None admin] RBAC: Authorizing `identity:get_project(project_id=b57372dfc633444d9f127657ca57d59c)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:17:17.674804 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c334b810-69e8-4fce-b4ce-7f84ce7f040c None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:17:17.677955 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 196/410] 10.222.0.30 () {56 vars in 1186 bytes} [Tue Sep 6 04:17:17 2022] GET /identity/v3/projects/b57372dfc633444d9f127657ca57d59c => generated 302 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:17:17.684174 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c334b810-69e8-4fce-b4ce-7f84ce7f040c None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:17.746831 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 215/411] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Sep 6 04:17:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 04:17:17.748574 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deaaf407-940f-4718-a307-7a69863da096 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:17.783955 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:17:17.864730 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c334b810-69e8-4fce-b4ce-7f84ce7f040c None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:17.880621 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deaaf407-940f-4718-a307-7a69863da096 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:17.906559 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf93fd3d-4f23-4a33-a00a-51db8800de86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:17.907601 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf93fd3d-4f23-4a33-a00a-51db8800de86 None admin] REQUEST_METHOD: `PUT` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:17.908074 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf93fd3d-4f23-4a33-a00a-51db8800de86 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:17.908538 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf93fd3d-4f23-4a33-a00a-51db8800de86 None admin] PATH_INFO: `/v3/projects/b57372dfc633444d9f127657ca57d59c/groups/60fbab6f26044826b072daa5861abbb4/roles/77197446dd2541b9a4fab9a654c08fc9` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:17.913292 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf93fd3d-4f23-4a33-a00a-51db8800de86 None admin] RBAC: Authorizing `identity:create_grant(project_id=b57372dfc633444d9f127657ca57d59c, group_id=60fbab6f26044826b072daa5861abbb4, role_id=77197446dd2541b9a4fab9a654c08fc9)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:17:17.915162 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf93fd3d-4f23-4a33-a00a-51db8800de86 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:17:17.922542 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2860f23f-eff2-4ee2-b5b7-2e4a11aee222 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:17.923241 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2860f23f-eff2-4ee2-b5b7-2e4a11aee222 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:17.923522 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2860f23f-eff2-4ee2-b5b7-2e4a11aee222 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:17.923796 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2860f23f-eff2-4ee2-b5b7-2e4a11aee222 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:17.938663 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2860f23f-eff2-4ee2-b5b7-2e4a11aee222 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 04:17:17.943909 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-2860f23f-eff2-4ee2-b5b7-2e4a11aee222 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 04:17:17.943909 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 04:17:17.943909 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 04:17:17.943909 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 04:17:17.943909 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 04:17:17.943909 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 04:17:17.943909 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 04:17:17.943909 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 04:17:17.943909 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 04:17:17.943909 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 04:17:17.943909 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 04:17:17.943909 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 04:17:17.943909 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 04:17:17.943909 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 04:17:17.943909 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 04:17:17.943909 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 04:17:17.944810 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 04:17:17.944810 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 04:17:17.944810 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 04:17:17.944810 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 04:17:17.944810 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 04:17:17.944810 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 04:17:17.944810 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 04:17:17.944810 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 04:17:17.944810 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 04:17:17.944810 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 04:17:17.944810 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 04:17:17.944810 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 04:17:17.944810 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 04:17:17.944810 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 04:17:17.944810 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 04:17:17.944810 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 04:17:17.946229 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 04:17:17.946229 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 04:17:17.946229 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 04:17:17.946229 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 04:17:17.946229 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 04:17:17.946229 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 04:17:17.946229 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 04:17:17.946229 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 04:17:17.946229 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 04:17:17.946229 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 04:17:17.946229 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 04:17:17.946229 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 04:17:17.946229 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 216/412] 10.222.0.30 () {56 vars in 1096 bytes} [Tue Sep 6 04:17:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 04:17:17.951011 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2860f23f-eff2-4ee2-b5b7-2e4a11aee222 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:17.995479 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 197/413] 10.222.0.30 () {58 vars in 1442 bytes} [Tue Sep 6 04:17:17 2022] PUT /identity/v3/projects/b57372dfc633444d9f127657ca57d59c/groups/60fbab6f26044826b072daa5861abbb4/roles/77197446dd2541b9a4fab9a654c08fc9 => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 04:17:18.055699 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:17:18.058759 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf93fd3d-4f23-4a33-a00a-51db8800de86 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:18.073723 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2860f23f-eff2-4ee2-b5b7-2e4a11aee222 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:18.105026 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:17:18.134741 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd7c5d5-2656-40a5-abd0-66bae3358a88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:18.135389 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd7c5d5-2656-40a5-abd0-66bae3358a88 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:18.135667 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd7c5d5-2656-40a5-abd0-66bae3358a88 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:18.135938 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd7c5d5-2656-40a5-abd0-66bae3358a88 None admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:18.136625 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd7c5d5-2656-40a5-abd0-66bae3358a88 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:17:18.136934 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd7c5d5-2656-40a5-abd0-66bae3358a88 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:17:18.139007 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd7c5d5-2656-40a5-abd0-66bae3358a88 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:17:18.160392 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 217/414] 10.222.0.30 () {56 vars in 1099 bytes} [Tue Sep 6 04:17:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 04:17:18.169124 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd7c5d5-2656-40a5-abd0-66bae3358a88 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:18.208689 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf93fd3d-4f23-4a33-a00a-51db8800de86 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:18.247821 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf0f15c-3993-47e5-8e68-f905c0a167a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:18.248818 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf0f15c-3993-47e5-8e68-f905c0a167a2 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:18.249293 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf0f15c-3993-47e5-8e68-f905c0a167a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:18.249786 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf0f15c-3993-47e5-8e68-f905c0a167a2 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:18.262522 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf0f15c-3993-47e5-8e68-f905c0a167a2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 04:17:18.267705 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-abf0f15c-3993-47e5-8e68-f905c0a167a2 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 06 04:17:18.267705 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 04:17:18.267705 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 04:17:18.267705 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 04:17:18.267705 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 04:17:18.267705 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 04:17:18.267705 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 04:17:18.267705 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 04:17:18.267705 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 04:17:18.267705 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 04:17:18.267705 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 04:17:18.267705 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 04:17:18.267705 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 04:17:18.267705 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 04:17:18.267705 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 04:17:18.267705 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 04:17:18.268580 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 04:17:18.268580 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 04:17:18.268580 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 04:17:18.268580 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 04:17:18.268580 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 04:17:18.268580 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 04:17:18.268580 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 04:17:18.268580 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 04:17:18.268580 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 04:17:18.268580 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 04:17:18.268580 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 04:17:18.268580 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 04:17:18.268580 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 04:17:18.268580 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 04:17:18.268580 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 04:17:18.268580 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 04:17:18.269400 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 04:17:18.269400 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 04:17:18.269400 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 04:17:18.269400 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 04:17:18.269400 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 04:17:18.269400 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 04:17:18.269400 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 04:17:18.269400 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 04:17:18.269400 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 04:17:18.269400 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 04:17:18.269400 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 06 04:17:18.269400 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 04:17:18.272437 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 198/415] 10.222.0.30 () {56 vars in 1099 bytes} [Tue Sep 6 04:17:18 2022] GET /identity/v3/roles/member => generated 84 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 04:17:18.277923 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf0f15c-3993-47e5-8e68-f905c0a167a2 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:18.299376 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd7c5d5-2656-40a5-abd0-66bae3358a88 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:18.337753 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb18ab2-798e-442c-bf75-af319681e01b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:18.338119 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb18ab2-798e-442c-bf75-af319681e01b None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:18.338407 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb18ab2-798e-442c-bf75-af319681e01b None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:18.338682 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb18ab2-798e-442c-bf75-af319681e01b None admin] PATH_INFO: `/v3/groups/00b3acdc6a4443d6904694fcb386f700` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:18.339962 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb18ab2-798e-442c-bf75-af319681e01b None admin] RBAC: Authorizing `identity:get_group(group_id=00b3acdc6a4443d6904694fcb386f700)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:17:18.341459 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb18ab2-798e-442c-bf75-af319681e01b None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:17:18.343656 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 218/416] 10.222.0.30 () {56 vars in 1180 bytes} [Tue Sep 6 04:17:18 2022] GET /identity/v3/groups/00b3acdc6a4443d6904694fcb386f700 => generated 229 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:17:18.349625 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb18ab2-798e-442c-bf75-af319681e01b None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:18.387009 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:17:18.398120 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:17:18.404470 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf0f15c-3993-47e5-8e68-f905c0a167a2 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:18.448764 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6333a89-8076-4390-b465-badc21219ec9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:18.450150 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6333a89-8076-4390-b465-badc21219ec9 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:18.451010 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6333a89-8076-4390-b465-badc21219ec9 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:18.451955 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6333a89-8076-4390-b465-badc21219ec9 None admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:18.452743 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6333a89-8076-4390-b465-badc21219ec9 None admin] RBAC: Adding query filter params (name=member) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:17:18.453513 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6333a89-8076-4390-b465-badc21219ec9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:17:18.455695 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6333a89-8076-4390-b465-badc21219ec9 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:17:18.475299 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 199/417] 10.222.0.30 () {56 vars in 1101 bytes} [Tue Sep 6 04:17:18 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:17:18.480257 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6333a89-8076-4390-b465-badc21219ec9 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:18.591325 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb18ab2-798e-442c-bf75-af319681e01b None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:18.640140 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435fe99e-13a3-4934-a10c-9997adeaaadd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:18.640758 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435fe99e-13a3-4934-a10c-9997adeaaadd None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:18.641037 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435fe99e-13a3-4934-a10c-9997adeaaadd None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:18.641321 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435fe99e-13a3-4934-a10c-9997adeaaadd None admin] PATH_INFO: `/v3/projects/8b3067b612904279b76189015d91b056` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:18.643801 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435fe99e-13a3-4934-a10c-9997adeaaadd None admin] RBAC: Authorizing `identity:get_project(project_id=8b3067b612904279b76189015d91b056)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:17:18.645334 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435fe99e-13a3-4934-a10c-9997adeaaadd None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:17:18.648392 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 219/418] 10.222.0.30 () {56 vars in 1186 bytes} [Tue Sep 6 04:17:18 2022] GET /identity/v3/projects/8b3067b612904279b76189015d91b056 => generated 344 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 04:17:18.652038 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6333a89-8076-4390-b465-badc21219ec9 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:18.654989 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435fe99e-13a3-4934-a10c-9997adeaaadd None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:18.712641 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6606def-7544-4aa8-8571-fd8c9b3d4c31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:18.713266 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6606def-7544-4aa8-8571-fd8c9b3d4c31 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:18.713574 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6606def-7544-4aa8-8571-fd8c9b3d4c31 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:18.713842 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6606def-7544-4aa8-8571-fd8c9b3d4c31 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:18.773658 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:17:18.788163 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:17:18.805589 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435fe99e-13a3-4934-a10c-9997adeaaadd None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:18.855991 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6606def-7544-4aa8-8571-fd8c9b3d4c31 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:17:18.857406 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6606def-7544-4aa8-8571-fd8c9b3d4c31 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:17:18.874017 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9788db31-8975-4f4c-8b1d-bfb76ee3e9f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:18.874694 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9788db31-8975-4f4c-8b1d-bfb76ee3e9f2 None admin] REQUEST_METHOD: `PUT` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:18.875029 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9788db31-8975-4f4c-8b1d-bfb76ee3e9f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:18.875249 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9788db31-8975-4f4c-8b1d-bfb76ee3e9f2 None admin] PATH_INFO: `/v3/projects/8b3067b612904279b76189015d91b056/groups/00b3acdc6a4443d6904694fcb386f700/roles/ea4c814713b149108e488de855b025e4` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:18.881330 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9788db31-8975-4f4c-8b1d-bfb76ee3e9f2 None admin] RBAC: Authorizing `identity:create_grant(project_id=8b3067b612904279b76189015d91b056, group_id=00b3acdc6a4443d6904694fcb386f700, role_id=ea4c814713b149108e488de855b025e4)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:17:18.882706 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9788db31-8975-4f4c-8b1d-bfb76ee3e9f2 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:17:18.956901 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 220/419] 10.222.0.30 () {58 vars in 1442 bytes} [Tue Sep 6 04:17:18 2022] PUT /identity/v3/projects/8b3067b612904279b76189015d91b056/groups/00b3acdc6a4443d6904694fcb386f700/roles/ea4c814713b149108e488de855b025e4 => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 04:17:18.990788 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 200/420] 10.222.0.30 () {58 vars in 1282 bytes} [Tue Sep 6 04:17:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 511 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 06 04:17:18.996295 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9788db31-8975-4f4c-8b1d-bfb76ee3e9f2 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:19.164113 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9788db31-8975-4f4c-8b1d-bfb76ee3e9f2 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:19.203431 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a75e46-dca9-40d9-ba47-1d357621f70c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:19.204440 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a75e46-dca9-40d9-ba47-1d357621f70c None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:19.204928 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a75e46-dca9-40d9-ba47-1d357621f70c None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:19.205561 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a75e46-dca9-40d9-ba47-1d357621f70c None admin] PATH_INFO: `/v3/projects/a88c6977d0554a1eab3e30118ede2f46` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:19.208129 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a75e46-dca9-40d9-ba47-1d357621f70c None admin] RBAC: Authorizing `identity:get_project(project_id=a88c6977d0554a1eab3e30118ede2f46)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:17:19.210751 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a75e46-dca9-40d9-ba47-1d357621f70c None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:17:19.213914 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 221/421] 10.222.0.30 () {56 vars in 1186 bytes} [Tue Sep 6 04:17:18 2022] GET /identity/v3/projects/a88c6977d0554a1eab3e30118ede2f46 => generated 298 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:17:19.219190 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6606def-7544-4aa8-8571-fd8c9b3d4c31 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:19.307609 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:17:19.357745 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6606def-7544-4aa8-8571-fd8c9b3d4c31 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:19.408237 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfce7f1f-1aca-4898-a5bb-84829a091b60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:19.408873 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfce7f1f-1aca-4898-a5bb-84829a091b60 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:19.409154 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfce7f1f-1aca-4898-a5bb-84829a091b60 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:19.409453 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfce7f1f-1aca-4898-a5bb-84829a091b60 None admin] PATH_INFO: `/v3/groups/60fbab6f26044826b072daa5861abbb4` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:19.411430 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfce7f1f-1aca-4898-a5bb-84829a091b60 None admin] RBAC: Authorizing `identity:get_group(group_id=60fbab6f26044826b072daa5861abbb4)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:17:19.412885 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfce7f1f-1aca-4898-a5bb-84829a091b60 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:17:19.415183 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 201/422] 10.222.0.30 () {56 vars in 1180 bytes} [Tue Sep 6 04:17:19 2022] GET /identity/v3/groups/60fbab6f26044826b072daa5861abbb4 => generated 226 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:17:19.420481 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a75e46-dca9-40d9-ba47-1d357621f70c None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:19.449765 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:17:19.548786 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6efebef-3cf0-4ed0-b74d-ff9d58e348b7 None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:19.549125 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6efebef-3cf0-4ed0-b74d-ff9d58e348b7 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:19.549476 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6efebef-3cf0-4ed0-b74d-ff9d58e348b7 None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:19.550947 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 202/423] 10.222.0.30 () {54 vars in 927 bytes} [Tue Sep 6 04:17:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 06 04:17:19.556945 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3904397-5c86-4e8c-8e78-b487c9e16295 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:19.557229 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3904397-5c86-4e8c-8e78-b487c9e16295 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:19.557532 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3904397-5c86-4e8c-8e78-b487c9e16295 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:19.578534 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-a3904397-5c86-4e8c-8e78-b487c9e16295 None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:17:19.580032 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-a3904397-5c86-4e8c-8e78-b487c9e16295 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:17:19.589073 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a75e46-dca9-40d9-ba47-1d357621f70c None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:19.636304 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e91855-77a5-4dd2-a06f-a86645ad5492 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:19.636927 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e91855-77a5-4dd2-a06f-a86645ad5492 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:19.637532 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e91855-77a5-4dd2-a06f-a86645ad5492 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:19.637904 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e91855-77a5-4dd2-a06f-a86645ad5492 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:19.638863 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e91855-77a5-4dd2-a06f-a86645ad5492 None admin] RBAC: Adding query filter params (group.id=60fbab6f26044826b072daa5861abbb4, role.id=29607b2de39f40d68bb6ceef64b8a732, scope.project.id=a88c6977d0554a1eab3e30118ede2f46) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:17:19.639269 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e91855-77a5-4dd2-a06f-a86645ad5492 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:17:19.640863 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e91855-77a5-4dd2-a06f-a86645ad5492 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:17:19.648588 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 222/424] 10.222.0.30 () {56 vars in 1376 bytes} [Tue Sep 6 04:17:19 2022] GET /identity/v3/role_assignments?group.id=60fbab6f26044826b072daa5861abbb4&scope.project.id=a88c6977d0554a1eab3e30118ede2f46&role.id=29607b2de39f40d68bb6ceef64b8a732 => generated 611 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:17:19.745102 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 203/425] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Sep 6 04:17:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 06 04:17:20.061850 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083bb72f-e6a1-4fe3-8ade-ccdcfd72a677 None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:20.062225 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083bb72f-e6a1-4fe3-8ade-ccdcfd72a677 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:20.062399 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083bb72f-e6a1-4fe3-8ade-ccdcfd72a677 None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:20.063745 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 223/426] 10.222.0.30 () {54 vars in 927 bytes} [Tue Sep 6 04:17:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 04:17:20.069354 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083bb72f-e6a1-4fe3-8ade-ccdcfd72a677 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:20.070725 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b122d559-4f43-4c8f-94bc-1ed77bd40f95 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:20.071158 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b122d559-4f43-4c8f-94bc-1ed77bd40f95 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:20.071317 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b122d559-4f43-4c8f-94bc-1ed77bd40f95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:20.097289 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-b122d559-4f43-4c8f-94bc-1ed77bd40f95 None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:17:20.098847 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-b122d559-4f43-4c8f-94bc-1ed77bd40f95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:17:20.165462 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:17:20.172921 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:17:20.187565 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 204/427] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Sep 6 04:17:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 04:17:20.198891 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083bb72f-e6a1-4fe3-8ade-ccdcfd72a677 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:20.237099 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883c2b0b-2da2-4487-80ce-da18b8dc512f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:20.237776 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883c2b0b-2da2-4487-80ce-da18b8dc512f None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:20.238058 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883c2b0b-2da2-4487-80ce-da18b8dc512f None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:20.238332 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883c2b0b-2da2-4487-80ce-da18b8dc512f None admin] PATH_INFO: `/v3/roles/member` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:20.250756 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883c2b0b-2da2-4487-80ce-da18b8dc512f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 04:17:20.256543 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-883c2b0b-2da2-4487-80ce-da18b8dc512f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 06 04:17:20.256543 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 04:17:20.256543 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 04:17:20.256543 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 04:17:20.256543 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 04:17:20.256543 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 04:17:20.256543 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 04:17:20.256543 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 04:17:20.256543 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 04:17:20.256543 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 04:17:20.256543 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 04:17:20.256543 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 04:17:20.256543 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 04:17:20.256543 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 04:17:20.256543 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 04:17:20.256543 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 04:17:20.257470 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 04:17:20.257470 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 04:17:20.257470 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 04:17:20.257470 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 04:17:20.257470 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 04:17:20.257470 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 04:17:20.257470 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 04:17:20.257470 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 04:17:20.257470 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 04:17:20.257470 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 04:17:20.257470 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 04:17:20.257470 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 04:17:20.257470 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 04:17:20.257470 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 04:17:20.257470 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 04:17:20.257470 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 04:17:20.258377 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 04:17:20.258377 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 04:17:20.258377 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 04:17:20.258377 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 04:17:20.258377 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 04:17:20.258377 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 04:17:20.258377 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 04:17:20.258377 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 04:17:20.258377 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 04:17:20.258377 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 04:17:20.258377 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 06 04:17:20.258377 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 04:17:20.258377 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 224/428] 10.222.0.30 () {56 vars in 1099 bytes} [Tue Sep 6 04:17:20 2022] GET /identity/v3/roles/member => generated 84 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 04:17:20.264841 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b122d559-4f43-4c8f-94bc-1ed77bd40f95 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:20.382101 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b122d559-4f43-4c8f-94bc-1ed77bd40f95 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:20.422196 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934a82b7-7d85-492c-8a86-1e2782784f6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:20.423045 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934a82b7-7d85-492c-8a86-1e2782784f6d None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:20.423459 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934a82b7-7d85-492c-8a86-1e2782784f6d None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:20.423866 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934a82b7-7d85-492c-8a86-1e2782784f6d None admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:20.424693 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-934a82b7-7d85-492c-8a86-1e2782784f6d None admin] RBAC: Adding query filter params (name=member) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:17:20.425131 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-934a82b7-7d85-492c-8a86-1e2782784f6d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:17:20.426693 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-934a82b7-7d85-492c-8a86-1e2782784f6d None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:17:20.438690 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 205/429] 10.222.0.30 () {56 vars in 1101 bytes} [Tue Sep 6 04:17:20 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:17:20.445081 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883c2b0b-2da2-4487-80ce-da18b8dc512f None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:20.506119 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a46397-e374-4db8-b409-7ce828f37096 None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:20.506853 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a46397-e374-4db8-b409-7ce828f37096 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:20.507442 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a46397-e374-4db8-b409-7ce828f37096 None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:20.509962 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 206/430] 10.222.0.30 () {54 vars in 927 bytes} [Tue Sep 6 04:17:20 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 06 04:17:20.511671 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a46397-e374-4db8-b409-7ce828f37096 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:20.575773 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883c2b0b-2da2-4487-80ce-da18b8dc512f None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:20.576372 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:17:20.620775 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b83af5-2f78-4e94-b390-a37c418d1ec0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:20.620775 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b83af5-2f78-4e94-b390-a37c418d1ec0 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:20.620775 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b83af5-2f78-4e94-b390-a37c418d1ec0 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:20.620775 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b83af5-2f78-4e94-b390-a37c418d1ec0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:20.624951 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:17:20.649754 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a46397-e374-4db8-b409-7ce828f37096 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:20.685685 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfcce3f0-adb1-40c3-9afd-eeff250cb448 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:20.686438 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcce3f0-adb1-40c3-9afd-eeff250cb448 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:20.686719 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcce3f0-adb1-40c3-9afd-eeff250cb448 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:20.686988 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcce3f0-adb1-40c3-9afd-eeff250cb448 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:20.697566 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfcce3f0-adb1-40c3-9afd-eeff250cb448 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 04:17:20.698751 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b83af5-2f78-4e94-b390-a37c418d1ec0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:17:20.700272 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b83af5-2f78-4e94-b390-a37c418d1ec0 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:17:20.703012 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-dfcce3f0-adb1-40c3-9afd-eeff250cb448 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 06 04:17:20.703012 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 04:17:20.703012 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 04:17:20.703012 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 04:17:20.703012 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 04:17:20.703012 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 04:17:20.703012 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 04:17:20.703012 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 04:17:20.703012 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 04:17:20.703012 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 04:17:20.703012 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 04:17:20.703012 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 04:17:20.703012 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 04:17:20.703012 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 04:17:20.703012 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 04:17:20.703012 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 04:17:20.703911 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 04:17:20.703911 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 04:17:20.703911 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 04:17:20.703911 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 04:17:20.703911 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 04:17:20.703911 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 04:17:20.703911 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 04:17:20.703911 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 04:17:20.703911 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 04:17:20.703911 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 04:17:20.703911 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 04:17:20.703911 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 04:17:20.703911 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 04:17:20.703911 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 04:17:20.703911 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 04:17:20.703911 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 04:17:20.705016 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 04:17:20.705016 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 04:17:20.705016 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 04:17:20.705016 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 04:17:20.705016 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 04:17:20.705016 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 04:17:20.705016 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 04:17:20.705016 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 04:17:20.705016 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 04:17:20.705016 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 04:17:20.705016 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 06 04:17:20.705016 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 04:17:20.705016 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 207/431] 10.222.0.30 () {56 vars in 1114 bytes} [Tue Sep 6 04:17:20 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 04:17:20.705883 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6f7486-f3f7-4db3-95d8-fa262a4a0b13 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:20.706136 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6f7486-f3f7-4db3-95d8-fa262a4a0b13 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:20.706416 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6f7486-f3f7-4db3-95d8-fa262a4a0b13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:20.724992 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-da6f7486-f3f7-4db3-95d8-fa262a4a0b13 None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:17:20.726226 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-da6f7486-f3f7-4db3-95d8-fa262a4a0b13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:17:20.803824 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 208/432] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Sep 6 04:17:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 04:17:20.804679 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6f7486-f3f7-4db3-95d8-fa262a4a0b13 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:20.832269 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 225/433] 10.222.0.30 () {58 vars in 1282 bytes} [Tue Sep 6 04:17:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 06 04:17:20.840473 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b83af5-2f78-4e94-b390-a37c418d1ec0 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:20.857934 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:17:20.887064 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:17:20.946569 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6f7486-f3f7-4db3-95d8-fa262a4a0b13 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:20.959997 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b83af5-2f78-4e94-b390-a37c418d1ec0 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:20.993412 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8ea128-2f53-4950-aa2e-2b2e111fb02e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:20.994565 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8ea128-2f53-4950-aa2e-2b2e111fb02e None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:20.995137 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8ea128-2f53-4950-aa2e-2b2e111fb02e None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:20.995393 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d139d167-77e5-4db5-84d5-297187c53376 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:20.995833 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8ea128-2f53-4950-aa2e-2b2e111fb02e None admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:20.996024 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d139d167-77e5-4db5-84d5-297187c53376 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:20.996302 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d139d167-77e5-4db5-84d5-297187c53376 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:20.996571 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d139d167-77e5-4db5-84d5-297187c53376 None admin] PATH_INFO: `/v3/projects/b57372dfc633444d9f127657ca57d59c` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:20.997436 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8ea128-2f53-4950-aa2e-2b2e111fb02e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:17:20.997717 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d139d167-77e5-4db5-84d5-297187c53376 None admin] RBAC: Authorizing `identity:get_project(project_id=b57372dfc633444d9f127657ca57d59c)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:17:20.998139 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8ea128-2f53-4950-aa2e-2b2e111fb02e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:17:20.999330 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d139d167-77e5-4db5-84d5-297187c53376 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:17:20.999923 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8ea128-2f53-4950-aa2e-2b2e111fb02e None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:17:21.005788 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 226/434] 10.222.0.30 () {56 vars in 1186 bytes} [Tue Sep 6 04:17:20 2022] GET /identity/v3/projects/b57372dfc633444d9f127657ca57d59c => generated 302 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:17:21.007277 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d139d167-77e5-4db5-84d5-297187c53376 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:21.010853 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 209/435] 10.222.0.30 () {56 vars in 1111 bytes} [Tue Sep 6 04:17:20 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:17:21.018391 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8ea128-2f53-4950-aa2e-2b2e111fb02e None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:21.117750 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d139d167-77e5-4db5-84d5-297187c53376 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:21.120065 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:17:21.125783 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:17:21.129776 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8ea128-2f53-4950-aa2e-2b2e111fb02e None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:21.153236 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f2dbc4-a4f2-4eae-b798-a1c67a585a68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:21.153913 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f2dbc4-a4f2-4eae-b798-a1c67a585a68 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:21.154199 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f2dbc4-a4f2-4eae-b798-a1c67a585a68 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:21.154449 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f2dbc4-a4f2-4eae-b798-a1c67a585a68 None admin] PATH_INFO: `/v3/groups/60fbab6f26044826b072daa5861abbb4` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:21.155623 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f2dbc4-a4f2-4eae-b798-a1c67a585a68 None admin] RBAC: Authorizing `identity:get_group(group_id=60fbab6f26044826b072daa5861abbb4)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:17:21.157131 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f2dbc4-a4f2-4eae-b798-a1c67a585a68 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:17:21.165782 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 227/436] 10.222.0.30 () {56 vars in 1180 bytes} [Tue Sep 6 04:17:21 2022] GET /identity/v3/groups/60fbab6f26044826b072daa5861abbb4 => generated 226 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 04:17:21.166754 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e597a224-505b-4b82-a9f3-410a954b47fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:21.167383 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e597a224-505b-4b82-a9f3-410a954b47fe None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:21.167668 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e597a224-505b-4b82-a9f3-410a954b47fe None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:21.167858 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f2dbc4-a4f2-4eae-b798-a1c67a585a68 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:21.167977 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e597a224-505b-4b82-a9f3-410a954b47fe None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:21.243172 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e597a224-505b-4b82-a9f3-410a954b47fe None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:17:21.244537 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e597a224-505b-4b82-a9f3-410a954b47fe None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:17:21.280812 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f2dbc4-a4f2-4eae-b798-a1c67a585a68 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:21.315281 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1beb01-e597-4eb7-8a83-2e21f9acfccc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:21.315917 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1beb01-e597-4eb7-8a83-2e21f9acfccc None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:21.316203 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1beb01-e597-4eb7-8a83-2e21f9acfccc None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:21.316476 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1beb01-e597-4eb7-8a83-2e21f9acfccc None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:21.327443 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1beb01-e597-4eb7-8a83-2e21f9acfccc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 04:17:21.332334 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-1c1beb01-e597-4eb7-8a83-2e21f9acfccc None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 06 04:17:21.332334 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 04:17:21.332334 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 04:17:21.332334 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 04:17:21.332334 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 04:17:21.332334 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 04:17:21.332334 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 04:17:21.332334 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 04:17:21.332334 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 04:17:21.332334 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 04:17:21.332334 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 04:17:21.332334 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 04:17:21.332334 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 04:17:21.332334 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 04:17:21.332334 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 04:17:21.332334 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 04:17:21.333594 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 04:17:21.333594 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 04:17:21.333594 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 04:17:21.333594 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 04:17:21.333594 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 04:17:21.333594 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 04:17:21.333594 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 04:17:21.333594 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 04:17:21.333594 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 04:17:21.333594 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 04:17:21.333594 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 04:17:21.333594 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 04:17:21.333594 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 04:17:21.333594 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 04:17:21.333594 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 04:17:21.333594 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 04:17:21.334418 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 04:17:21.334418 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 04:17:21.334418 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 04:17:21.334418 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 04:17:21.334418 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 04:17:21.334418 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 04:17:21.334418 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 04:17:21.334418 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 04:17:21.334418 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 04:17:21.334418 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 04:17:21.334418 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 06 04:17:21.334418 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 04:17:21.334418 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 228/437] 10.222.0.30 () {56 vars in 1114 bytes} [Tue Sep 6 04:17:21 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 04:17:21.334418 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1beb01-e597-4eb7-8a83-2e21f9acfccc None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:21.341479 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:17:21.350752 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 210/438] 10.222.0.30 () {58 vars in 1282 bytes} [Tue Sep 6 04:17:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 06 04:17:21.350752 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d3b107-9442-4f83-ac2e-d9e72b753acf None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:21.350752 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d3b107-9442-4f83-ac2e-d9e72b753acf None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:21.350752 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d3b107-9442-4f83-ac2e-d9e72b753acf None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:21.350752 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 211/439] 10.222.0.30 () {54 vars in 927 bytes} [Tue Sep 6 04:17:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 04:17:21.350752 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d3b107-9442-4f83-ac2e-d9e72b753acf None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:21.371194 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:17:21.439714 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1beb01-e597-4eb7-8a83-2e21f9acfccc None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:21.453966 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d3b107-9442-4f83-ac2e-d9e72b753acf None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:21.472223 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3cfd6de-bf47-4118-9058-9e259e8bf806 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:21.472819 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cfd6de-bf47-4118-9058-9e259e8bf806 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:21.473668 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cfd6de-bf47-4118-9058-9e259e8bf806 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:21.474015 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cfd6de-bf47-4118-9058-9e259e8bf806 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:21.474646 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3cfd6de-bf47-4118-9058-9e259e8bf806 None admin] RBAC: Adding query filter params (group.id=60fbab6f26044826b072daa5861abbb4, role.id=29607b2de39f40d68bb6ceef64b8a732, scope.project.id=b57372dfc633444d9f127657ca57d59c) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:17:21.474943 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3cfd6de-bf47-4118-9058-9e259e8bf806 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:17:21.476382 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3cfd6de-bf47-4118-9058-9e259e8bf806 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:17:21.482226 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 229/440] 10.222.0.30 () {56 vars in 1376 bytes} [Tue Sep 6 04:17:21 2022] GET /identity/v3/role_assignments?group.id=60fbab6f26044826b072daa5861abbb4&scope.project.id=b57372dfc633444d9f127657ca57d59c&role.id=29607b2de39f40d68bb6ceef64b8a732 => generated 611 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:17:21.482793 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3cfd6de-bf47-4118-9058-9e259e8bf806 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:21.496111 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2384fc67-4690-4abc-bc09-19bcae12b733 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:21.496793 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2384fc67-4690-4abc-bc09-19bcae12b733 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:21.497080 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2384fc67-4690-4abc-bc09-19bcae12b733 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:21.497386 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2384fc67-4690-4abc-bc09-19bcae12b733 None admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:21.498071 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2384fc67-4690-4abc-bc09-19bcae12b733 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:17:21.498382 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2384fc67-4690-4abc-bc09-19bcae12b733 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:17:21.499938 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2384fc67-4690-4abc-bc09-19bcae12b733 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:17:21.510359 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 212/441] 10.222.0.30 () {56 vars in 1111 bytes} [Tue Sep 6 04:17:21 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:17:21.511843 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd52f5d-f3ff-405d-9cbf-1aea023d9f2a None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:21.512127 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd52f5d-f3ff-405d-9cbf-1aea023d9f2a None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:21.512400 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd52f5d-f3ff-405d-9cbf-1aea023d9f2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:21.531890 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-0bd52f5d-f3ff-405d-9cbf-1aea023d9f2a None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:17:21.533242 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-0bd52f5d-f3ff-405d-9cbf-1aea023d9f2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:17:21.597352 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:17:21.597559 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:17:21.604386 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3cfd6de-bf47-4118-9058-9e259e8bf806 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:21.617765 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 213/442] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Sep 6 04:17:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 04:17:21.617765 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd52f5d-f3ff-405d-9cbf-1aea023d9f2a None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:21.659757 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30df84b-c184-4d9d-bb0d-b90ea1d13080 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:21.661109 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30df84b-c184-4d9d-bb0d-b90ea1d13080 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:21.661109 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30df84b-c184-4d9d-bb0d-b90ea1d13080 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:21.661109 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30df84b-c184-4d9d-bb0d-b90ea1d13080 None admin] PATH_INFO: `/v3/projects/a88c6977d0554a1eab3e30118ede2f46` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:21.662462 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30df84b-c184-4d9d-bb0d-b90ea1d13080 None admin] RBAC: Authorizing `identity:get_project(project_id=a88c6977d0554a1eab3e30118ede2f46)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:17:21.664434 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30df84b-c184-4d9d-bb0d-b90ea1d13080 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:17:21.667038 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 230/443] 10.222.0.30 () {56 vars in 1186 bytes} [Tue Sep 6 04:17:21 2022] GET /identity/v3/projects/a88c6977d0554a1eab3e30118ede2f46 => generated 298 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:17:21.672527 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30df84b-c184-4d9d-bb0d-b90ea1d13080 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:21.749484 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd52f5d-f3ff-405d-9cbf-1aea023d9f2a None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:21.785885 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4283fa8d-92ce-40a8-84c6-1a82f4264b35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:21.786359 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30df84b-c184-4d9d-bb0d-b90ea1d13080 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:21.786814 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4283fa8d-92ce-40a8-84c6-1a82f4264b35 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:21.786814 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4283fa8d-92ce-40a8-84c6-1a82f4264b35 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:21.787049 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4283fa8d-92ce-40a8-84c6-1a82f4264b35 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:21.821995 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c6f6ee-9cf2-459e-b62f-62bbf6e3a2f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:21.822622 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c6f6ee-9cf2-459e-b62f-62bbf6e3a2f7 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:21.822898 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c6f6ee-9cf2-459e-b62f-62bbf6e3a2f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:21.823167 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c6f6ee-9cf2-459e-b62f-62bbf6e3a2f7 None admin] PATH_INFO: `/v3/groups/60fbab6f26044826b072daa5861abbb4` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:21.824271 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c6f6ee-9cf2-459e-b62f-62bbf6e3a2f7 None admin] RBAC: Authorizing `identity:get_group(group_id=60fbab6f26044826b072daa5861abbb4)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:17:21.825799 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c6f6ee-9cf2-459e-b62f-62bbf6e3a2f7 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:17:21.827538 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 231/444] 10.222.0.30 () {56 vars in 1180 bytes} [Tue Sep 6 04:17:21 2022] GET /identity/v3/groups/60fbab6f26044826b072daa5861abbb4 => generated 226 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 04:17:21.833459 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c6f6ee-9cf2-459e-b62f-62bbf6e3a2f7 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:21.850706 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:17:21.860861 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4283fa8d-92ce-40a8-84c6-1a82f4264b35 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:17:21.862340 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4283fa8d-92ce-40a8-84c6-1a82f4264b35 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:17:21.866103 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:17:21.933996 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c6f6ee-9cf2-459e-b62f-62bbf6e3a2f7 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:21.963735 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 214/445] 10.222.0.30 () {58 vars in 1282 bytes} [Tue Sep 6 04:17:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 06 04:17:21.964752 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4283fa8d-92ce-40a8-84c6-1a82f4264b35 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:21.971667 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ea8aa9-083f-49c3-b742-9c5fec0e7782 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:21.972274 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ea8aa9-083f-49c3-b742-9c5fec0e7782 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:21.972552 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ea8aa9-083f-49c3-b742-9c5fec0e7782 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:21.972821 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ea8aa9-083f-49c3-b742-9c5fec0e7782 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:21.973615 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ea8aa9-083f-49c3-b742-9c5fec0e7782 None admin] RBAC: Adding query filter params (group.id=60fbab6f26044826b072daa5861abbb4, role.id=77197446dd2541b9a4fab9a654c08fc9, scope.project.id=a88c6977d0554a1eab3e30118ede2f46) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:17:21.973919 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ea8aa9-083f-49c3-b742-9c5fec0e7782 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:17:21.975345 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ea8aa9-083f-49c3-b742-9c5fec0e7782 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:17:21.981103 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 232/446] 10.222.0.30 () {56 vars in 1376 bytes} [Tue Sep 6 04:17:21 2022] GET /identity/v3/role_assignments?group.id=60fbab6f26044826b072daa5861abbb4&scope.project.id=a88c6977d0554a1eab3e30118ede2f46&role.id=77197446dd2541b9a4fab9a654c08fc9 => generated 611 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 04:17:21.986315 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ea8aa9-083f-49c3-b742-9c5fec0e7782 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:22.063107 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:17:22.074405 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4283fa8d-92ce-40a8-84c6-1a82f4264b35 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:22.093812 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:17:22.100597 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ea8aa9-083f-49c3-b742-9c5fec0e7782 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:22.113062 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d9fb34-776e-4bc9-9fb8-4b7b60749ace None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:22.114222 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d9fb34-776e-4bc9-9fb8-4b7b60749ace None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:22.114707 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d9fb34-776e-4bc9-9fb8-4b7b60749ace None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:22.115142 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d9fb34-776e-4bc9-9fb8-4b7b60749ace None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:22.128312 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d9fb34-776e-4bc9-9fb8-4b7b60749ace None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 04:17:22.133789 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-82d9fb34-776e-4bc9-9fb8-4b7b60749ace None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 04:17:22.133789 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 04:17:22.133789 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 04:17:22.133789 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 04:17:22.133789 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 04:17:22.133789 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 04:17:22.133789 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 04:17:22.133789 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 04:17:22.133789 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 04:17:22.133789 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 04:17:22.133789 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 04:17:22.133789 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 04:17:22.133789 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 04:17:22.133789 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 04:17:22.133789 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 04:17:22.133789 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 04:17:22.134703 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 04:17:22.134703 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 04:17:22.134703 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 04:17:22.134703 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 04:17:22.134703 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 04:17:22.134703 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 04:17:22.134703 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 04:17:22.134703 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 04:17:22.134703 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 04:17:22.134703 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 04:17:22.134703 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 04:17:22.134703 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 04:17:22.134703 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 04:17:22.134703 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 04:17:22.134703 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 04:17:22.134703 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 04:17:22.135519 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 04:17:22.135519 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 04:17:22.135519 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 04:17:22.135519 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 04:17:22.135519 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 04:17:22.135519 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 04:17:22.135519 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 04:17:22.135519 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 04:17:22.135519 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 04:17:22.135519 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 04:17:22.135519 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 04:17:22.135519 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 04:17:22.135519 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 215/447] 10.222.0.30 () {56 vars in 1096 bytes} [Tue Sep 6 04:17:21 2022] GET /identity/v3/roles/admin => generated 83 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 04:17:22.140573 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d9fb34-776e-4bc9-9fb8-4b7b60749ace None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:22.146162 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c19558-8c78-4a4e-be20-092db37cbb6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:22.146938 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c19558-8c78-4a4e-be20-092db37cbb6b None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:22.147208 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c19558-8c78-4a4e-be20-092db37cbb6b None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:22.147485 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c19558-8c78-4a4e-be20-092db37cbb6b None admin] PATH_INFO: `/v3/projects/b57372dfc633444d9f127657ca57d59c` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:22.149729 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c19558-8c78-4a4e-be20-092db37cbb6b None admin] RBAC: Authorizing `identity:get_project(project_id=b57372dfc633444d9f127657ca57d59c)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:17:22.151183 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c19558-8c78-4a4e-be20-092db37cbb6b None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:17:22.153123 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 233/448] 10.222.0.30 () {56 vars in 1186 bytes} [Tue Sep 6 04:17:21 2022] GET /identity/v3/projects/b57372dfc633444d9f127657ca57d59c => generated 302 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:17:22.159069 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c19558-8c78-4a4e-be20-092db37cbb6b None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:22.259808 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d9fb34-776e-4bc9-9fb8-4b7b60749ace None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:22.283217 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c19558-8c78-4a4e-be20-092db37cbb6b None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:22.300583 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f1e680-b7f5-4c09-b56e-40053b93f7a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:22.301762 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f1e680-b7f5-4c09-b56e-40053b93f7a5 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:22.302041 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f1e680-b7f5-4c09-b56e-40053b93f7a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:22.302370 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f1e680-b7f5-4c09-b56e-40053b93f7a5 None admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:22.303043 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f1e680-b7f5-4c09-b56e-40053b93f7a5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:17:22.303341 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f1e680-b7f5-4c09-b56e-40053b93f7a5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:17:22.304755 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f1e680-b7f5-4c09-b56e-40053b93f7a5 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:17:22.314062 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 216/449] 10.222.0.30 () {56 vars in 1099 bytes} [Tue Sep 6 04:17:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 04:17:22.320129 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f1e680-b7f5-4c09-b56e-40053b93f7a5 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:22.323768 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8093425f-0049-4a2b-b9e8-6e6ad56f48d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:22.324410 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8093425f-0049-4a2b-b9e8-6e6ad56f48d3 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:22.324694 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8093425f-0049-4a2b-b9e8-6e6ad56f48d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:22.324968 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8093425f-0049-4a2b-b9e8-6e6ad56f48d3 None admin] PATH_INFO: `/v3/groups/60fbab6f26044826b072daa5861abbb4` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:22.326149 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8093425f-0049-4a2b-b9e8-6e6ad56f48d3 None admin] RBAC: Authorizing `identity:get_group(group_id=60fbab6f26044826b072daa5861abbb4)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:17:22.327601 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8093425f-0049-4a2b-b9e8-6e6ad56f48d3 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:17:22.329370 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 234/450] 10.222.0.30 () {56 vars in 1180 bytes} [Tue Sep 6 04:17:22 2022] GET /identity/v3/groups/60fbab6f26044826b072daa5861abbb4 => generated 226 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:17:22.336454 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8093425f-0049-4a2b-b9e8-6e6ad56f48d3 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:22.341343 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:17:22.377014 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:17:22.451307 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f1e680-b7f5-4c09-b56e-40053b93f7a5 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:22.470659 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8093425f-0049-4a2b-b9e8-6e6ad56f48d3 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:22.491065 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa64a52-9416-4ee9-87b3-f41c00376579 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:22.491684 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa64a52-9416-4ee9-87b3-f41c00376579 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:22.491967 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa64a52-9416-4ee9-87b3-f41c00376579 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:22.492244 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa64a52-9416-4ee9-87b3-f41c00376579 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:22.512409 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e42fd5-c83d-432f-8339-c1e20c945d89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:22.513009 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e42fd5-c83d-432f-8339-c1e20c945d89 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:22.513283 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e42fd5-c83d-432f-8339-c1e20c945d89 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:22.513581 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e42fd5-c83d-432f-8339-c1e20c945d89 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:22.514304 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e42fd5-c83d-432f-8339-c1e20c945d89 None admin] RBAC: Adding query filter params (group.id=60fbab6f26044826b072daa5861abbb4, role.id=77197446dd2541b9a4fab9a654c08fc9, scope.project.id=b57372dfc633444d9f127657ca57d59c) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:17:22.514609 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e42fd5-c83d-432f-8339-c1e20c945d89 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:17:22.516065 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e42fd5-c83d-432f-8339-c1e20c945d89 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:17:22.522390 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 235/451] 10.222.0.30 () {56 vars in 1376 bytes} [Tue Sep 6 04:17:22 2022] GET /identity/v3/role_assignments?group.id=60fbab6f26044826b072daa5861abbb4&scope.project.id=b57372dfc633444d9f127657ca57d59c&role.id=77197446dd2541b9a4fab9a654c08fc9 => generated 611 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 04:17:22.562569 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa64a52-9416-4ee9-87b3-f41c00376579 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:17:22.564120 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa64a52-9416-4ee9-87b3-f41c00376579 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:17:22.581872 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:17:22.671225 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 217/452] 10.222.0.30 () {58 vars in 1282 bytes} [Tue Sep 6 04:17:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 06 04:17:22.678113 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e42fd5-c83d-432f-8339-c1e20c945d89 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:22.776900 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:17:22.782875 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e42fd5-c83d-432f-8339-c1e20c945d89 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:22.822660 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3199ae7b-b91a-480c-93f5-1429ac3ed425 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:22.823257 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3199ae7b-b91a-480c-93f5-1429ac3ed425 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:22.823499 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3199ae7b-b91a-480c-93f5-1429ac3ed425 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:22.823775 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3199ae7b-b91a-480c-93f5-1429ac3ed425 None admin] PATH_INFO: `/v3/projects/8b3067b612904279b76189015d91b056` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:22.824931 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3199ae7b-b91a-480c-93f5-1429ac3ed425 None admin] RBAC: Authorizing `identity:get_project(project_id=8b3067b612904279b76189015d91b056)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:17:22.826504 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3199ae7b-b91a-480c-93f5-1429ac3ed425 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:17:22.828138 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 236/453] 10.222.0.30 () {56 vars in 1186 bytes} [Tue Sep 6 04:17:22 2022] GET /identity/v3/projects/8b3067b612904279b76189015d91b056 => generated 344 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:17:22.834338 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa64a52-9416-4ee9-87b3-f41c00376579 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:22.933749 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa64a52-9416-4ee9-87b3-f41c00376579 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:22.947783 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:17:22.967188 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1433b5d-85ab-4f07-a588-240744fe8870 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:22.969406 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1433b5d-85ab-4f07-a588-240744fe8870 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:22.969406 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1433b5d-85ab-4f07-a588-240744fe8870 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:22.969406 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1433b5d-85ab-4f07-a588-240744fe8870 None admin] PATH_INFO: `/v3/groups/00b3acdc6a4443d6904694fcb386f700` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:22.969654 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1433b5d-85ab-4f07-a588-240744fe8870 None admin] RBAC: Authorizing `identity:get_group(group_id=00b3acdc6a4443d6904694fcb386f700)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:17:22.970891 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1433b5d-85ab-4f07-a588-240744fe8870 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:17:22.972490 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 218/454] 10.222.0.30 () {56 vars in 1180 bytes} [Tue Sep 6 04:17:22 2022] GET /identity/v3/groups/00b3acdc6a4443d6904694fcb386f700 => generated 229 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:17:22.978635 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3199ae7b-b91a-480c-93f5-1429ac3ed425 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:23.077474 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3199ae7b-b91a-480c-93f5-1429ac3ed425 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:23.110153 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9f8fd0-8050-4c5b-91c9-916d4850de44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:23.110753 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9f8fd0-8050-4c5b-91c9-916d4850de44 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:23.111035 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9f8fd0-8050-4c5b-91c9-916d4850de44 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:23.111306 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9f8fd0-8050-4c5b-91c9-916d4850de44 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:23.112035 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9f8fd0-8050-4c5b-91c9-916d4850de44 None admin] RBAC: Adding query filter params (group.id=00b3acdc6a4443d6904694fcb386f700, role.id=ea4c814713b149108e488de855b025e4, scope.project.id=8b3067b612904279b76189015d91b056) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:17:23.112344 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9f8fd0-8050-4c5b-91c9-916d4850de44 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:17:23.114439 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9f8fd0-8050-4c5b-91c9-916d4850de44 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:17:23.120031 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 237/455] 10.222.0.30 () {56 vars in 1376 bytes} [Tue Sep 6 04:17:22 2022] GET /identity/v3/role_assignments?group.id=00b3acdc6a4443d6904694fcb386f700&scope.project.id=8b3067b612904279b76189015d91b056&role.id=ea4c814713b149108e488de855b025e4 => generated 611 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:17:25.475309 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b33f29a-7273-426f-83a9-e7eb0ae89070 None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:25.475841 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b33f29a-7273-426f-83a9-e7eb0ae89070 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:25.476025 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b33f29a-7273-426f-83a9-e7eb0ae89070 None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:25.477620 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 219/456] 10.222.0.30 () {54 vars in 927 bytes} [Tue Sep 6 04:17:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 04:17:25.490491 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519aaa32-be28-4b80-860c-a1dfe2ba40e1 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:25.491015 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519aaa32-be28-4b80-860c-a1dfe2ba40e1 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:25.491015 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519aaa32-be28-4b80-860c-a1dfe2ba40e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:25.514732 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-519aaa32-be28-4b80-860c-a1dfe2ba40e1 None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:17:25.515948 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-519aaa32-be28-4b80-860c-a1dfe2ba40e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:17:25.548055 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:17:25.595283 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a7e2cc-74fa-4ac2-b31b-787021ba0c07 None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:25.595618 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a7e2cc-74fa-4ac2-b31b-787021ba0c07 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:25.595827 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a7e2cc-74fa-4ac2-b31b-787021ba0c07 None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:25.597379 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 220/457] 10.222.0.30 () {54 vars in 927 bytes} [Tue Sep 6 04:17:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 04:17:25.603527 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14497af4-b210-45a8-96a6-5a6b02925e47 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:25.603807 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14497af4-b210-45a8-96a6-5a6b02925e47 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:25.604081 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14497af4-b210-45a8-96a6-5a6b02925e47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:25.619489 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 238/458] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Sep 6 04:17:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 04:17:25.635275 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-14497af4-b210-45a8-96a6-5a6b02925e47 None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:17:25.636912 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-14497af4-b210-45a8-96a6-5a6b02925e47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:17:25.743815 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 221/459] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Sep 6 04:17:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 06 04:17:26.029531 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519aaa32-be28-4b80-860c-a1dfe2ba40e1 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:26.057656 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14497af4-b210-45a8-96a6-5a6b02925e47 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:26.144288 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519aaa32-be28-4b80-860c-a1dfe2ba40e1 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:26.180425 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87452b66-7553-4a00-b68b-3abe4f0930d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:26.181064 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87452b66-7553-4a00-b68b-3abe4f0930d9 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:26.181364 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87452b66-7553-4a00-b68b-3abe4f0930d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:26.181640 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87452b66-7553-4a00-b68b-3abe4f0930d9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:26.187964 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14497af4-b210-45a8-96a6-5a6b02925e47 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:26.192566 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87452b66-7553-4a00-b68b-3abe4f0930d9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:17:26.194220 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87452b66-7553-4a00-b68b-3abe4f0930d9 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:17:26.204263 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:17:26.212963 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-87452b66-7553-4a00-b68b-3abe4f0930d9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 04:17:26.212963 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 04:17:26.212963 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 04:17:26.212963 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 04:17:26.212963 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 04:17:26.212963 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 04:17:26.212963 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 04:17:26.212963 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 04:17:26.212963 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 04:17:26.212963 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 04:17:26.212963 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 04:17:26.212963 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 04:17:26.212963 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 04:17:26.212963 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 04:17:26.212963 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 04:17:26.212963 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 04:17:26.213952 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 04:17:26.213952 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 04:17:26.213952 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 04:17:26.213952 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 04:17:26.213952 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 04:17:26.213952 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 04:17:26.213952 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 04:17:26.213952 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 04:17:26.213952 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 04:17:26.213952 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 04:17:26.213952 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 04:17:26.213952 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 04:17:26.213952 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 04:17:26.213952 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 04:17:26.213952 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 04:17:26.213952 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 04:17:26.213952 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 04:17:26.215235 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 04:17:26.215235 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 04:17:26.215235 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 04:17:26.215235 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 239/460] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Sep 6 04:17:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 04:17:26.218685 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01953b68-7842-439e-9a14-3c34832751cf None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:26.218685 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01953b68-7842-439e-9a14-3c34832751cf None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:26.218685 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01953b68-7842-439e-9a14-3c34832751cf None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:26.227656 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 240/461] 10.222.0.30 () {54 vars in 927 bytes} [Tue Sep 6 04:17:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 04:17:26.227656 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01953b68-7842-439e-9a14-3c34832751cf None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:26.230768 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd4a418-7b87-400c-8d25-e98eeafa1d36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:26.231415 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd4a418-7b87-400c-8d25-e98eeafa1d36 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:26.231695 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd4a418-7b87-400c-8d25-e98eeafa1d36 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:26.232078 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd4a418-7b87-400c-8d25-e98eeafa1d36 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:26.235557 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:17:26.243654 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd4a418-7b87-400c-8d25-e98eeafa1d36 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:17:26.245470 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd4a418-7b87-400c-8d25-e98eeafa1d36 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:17:26.262497 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-1dd4a418-7b87-400c-8d25-e98eeafa1d36 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 04:17:26.262497 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 04:17:26.262497 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 04:17:26.262497 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 04:17:26.262497 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 04:17:26.262497 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 04:17:26.262497 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 04:17:26.262497 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 04:17:26.262497 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 04:17:26.262497 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 04:17:26.262497 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 04:17:26.262497 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 04:17:26.262497 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 04:17:26.262497 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 04:17:26.262497 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 04:17:26.262497 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 04:17:26.264690 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 04:17:26.264690 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 04:17:26.264690 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 04:17:26.264690 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 04:17:26.264690 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 04:17:26.264690 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 04:17:26.264690 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 04:17:26.264690 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 04:17:26.264690 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 04:17:26.264690 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 04:17:26.264690 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 04:17:26.264690 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 04:17:26.264690 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 04:17:26.264690 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 04:17:26.264690 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 04:17:26.264690 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 04:17:26.264690 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 04:17:26.266819 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 04:17:26.266819 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 04:17:26.266819 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 04:17:26.266819 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 222/462] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Sep 6 04:17:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 04:17:26.266819 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58e4349-1b5a-4f01-a64b-5dabf11c03e4 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:26.266819 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58e4349-1b5a-4f01-a64b-5dabf11c03e4 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:26.266819 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58e4349-1b5a-4f01-a64b-5dabf11c03e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:26.285552 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-e58e4349-1b5a-4f01-a64b-5dabf11c03e4 None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:17:26.286436 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-e58e4349-1b5a-4f01-a64b-5dabf11c03e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:17:26.331433 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01953b68-7842-439e-9a14-3c34832751cf None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:26.358533 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 223/463] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Sep 6 04:17:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 04:17:26.359474 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58e4349-1b5a-4f01-a64b-5dabf11c03e4 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:26.377479 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db4ebc4-fbea-4f44-b33a-c84198f54136 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:26.378364 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db4ebc4-fbea-4f44-b33a-c84198f54136 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:26.378875 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db4ebc4-fbea-4f44-b33a-c84198f54136 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:26.382172 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db4ebc4-fbea-4f44-b33a-c84198f54136 None admin] PATH_INFO: `/v3/domains` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:26.382172 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db4ebc4-fbea-4f44-b33a-c84198f54136 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:17:26.383218 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db4ebc4-fbea-4f44-b33a-c84198f54136 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:17:26.384836 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db4ebc4-fbea-4f44-b33a-c84198f54136 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:17:26.405699 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 241/464] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Sep 6 04:17:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 04:17:26.407334 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e65b39-1ed7-4ba6-b626-c184bbc9489e None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:26.407620 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e65b39-1ed7-4ba6-b626-c184bbc9489e None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:26.407892 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e65b39-1ed7-4ba6-b626-c184bbc9489e None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:26.409249 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 242/465] 10.222.0.30 () {54 vars in 927 bytes} [Tue Sep 6 04:17:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 04:17:26.411630 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e65b39-1ed7-4ba6-b626-c184bbc9489e None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:26.485735 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:17:26.493706 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58e4349-1b5a-4f01-a64b-5dabf11c03e4 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:26.513287 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:17:26.531363 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172ca43a-8a23-4afc-8d0b-f6512b9ea55a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:26.532625 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172ca43a-8a23-4afc-8d0b-f6512b9ea55a None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:26.533120 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172ca43a-8a23-4afc-8d0b-f6512b9ea55a None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:26.533752 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172ca43a-8a23-4afc-8d0b-f6512b9ea55a None admin] PATH_INFO: `/v3/domains` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:26.534957 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172ca43a-8a23-4afc-8d0b-f6512b9ea55a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:17:26.536076 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172ca43a-8a23-4afc-8d0b-f6512b9ea55a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:17:26.537703 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172ca43a-8a23-4afc-8d0b-f6512b9ea55a None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:17:26.554993 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e65b39-1ed7-4ba6-b626-c184bbc9489e None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:26.556028 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 224/466] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Sep 6 04:17:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:17:26.557504 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7cccbd-7118-455c-90a9-051553c081bc None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:26.558008 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7cccbd-7118-455c-90a9-051553c081bc None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:26.558489 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7cccbd-7118-455c-90a9-051553c081bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:26.581904 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-7e7cccbd-7118-455c-90a9-051553c081bc None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:17:26.583612 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-7e7cccbd-7118-455c-90a9-051553c081bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:17:26.604614 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73d7d70-ba4b-4594-927d-ffd3fe66af6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:26.605613 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73d7d70-ba4b-4594-927d-ffd3fe66af6a None admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:26.606142 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73d7d70-ba4b-4594-927d-ffd3fe66af6a None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:26.606616 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73d7d70-ba4b-4594-927d-ffd3fe66af6a None admin] PATH_INFO: `/v3/users` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:26.607513 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e73d7d70-ba4b-4594-927d-ffd3fe66af6a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:17:26.609193 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e73d7d70-ba4b-4594-927d-ffd3fe66af6a None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:17:26.664312 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 225/467] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Sep 6 04:17:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 04:17:26.665010 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7cccbd-7118-455c-90a9-051553c081bc None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:26.724952 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 243/468] 10.222.0.30 () {60 vars in 1132 bytes} [Tue Sep 6 04:17:26 2022] POST /identity/v3/users => generated 246 bytes in 315 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 04:17:26.726145 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73d7d70-ba4b-4594-927d-ffd3fe66af6a None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:26.794145 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:17:26.836318 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7cccbd-7118-455c-90a9-051553c081bc None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:26.906166 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311834f4-4ae5-43ee-aae7-5e613aa53e0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:26.907245 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311834f4-4ae5-43ee-aae7-5e613aa53e0a None admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:26.907537 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311834f4-4ae5-43ee-aae7-5e613aa53e0a None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:26.907815 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311834f4-4ae5-43ee-aae7-5e613aa53e0a None admin] PATH_INFO: `/v3/users` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:26.908479 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311834f4-4ae5-43ee-aae7-5e613aa53e0a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:17:26.910484 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311834f4-4ae5-43ee-aae7-5e613aa53e0a None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:17:26.939476 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73d7d70-ba4b-4594-927d-ffd3fe66af6a None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:26.973767 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:17:26.982556 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822acd06-3713-4a81-9ce7-729f921fbb8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:26.983201 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822acd06-3713-4a81-9ce7-729f921fbb8b None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:26.983719 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822acd06-3713-4a81-9ce7-729f921fbb8b None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:26.983864 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822acd06-3713-4a81-9ce7-729f921fbb8b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:26.992202 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 226/469] 10.222.0.30 () {60 vars in 1132 bytes} [Tue Sep 6 04:17:26 2022] POST /identity/v3/users => generated 248 bytes in 327 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 04:17:26.995479 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311834f4-4ae5-43ee-aae7-5e613aa53e0a None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:26.996519 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822acd06-3713-4a81-9ce7-729f921fbb8b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:17:26.998092 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822acd06-3713-4a81-9ce7-729f921fbb8b None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:17:27.014997 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-822acd06-3713-4a81-9ce7-729f921fbb8b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 04:17:27.014997 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 04:17:27.014997 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 04:17:27.014997 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 04:17:27.014997 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 04:17:27.014997 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 04:17:27.014997 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 04:17:27.014997 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 04:17:27.014997 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 04:17:27.014997 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 04:17:27.014997 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 04:17:27.014997 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 04:17:27.014997 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 04:17:27.014997 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 04:17:27.014997 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 04:17:27.014997 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 04:17:27.015995 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 04:17:27.015995 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 04:17:27.015995 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 04:17:27.015995 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 04:17:27.015995 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 04:17:27.015995 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 04:17:27.015995 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 04:17:27.015995 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 04:17:27.015995 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 04:17:27.015995 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 04:17:27.015995 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 04:17:27.015995 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 04:17:27.015995 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 04:17:27.015995 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 04:17:27.015995 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 04:17:27.015995 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 04:17:27.015995 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 04:17:27.016896 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 04:17:27.016896 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 04:17:27.016896 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 04:17:27.016896 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 244/470] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Sep 6 04:17:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 290 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 04:17:27.017618 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d705f7a-2bac-4542-8b94-38e486808afe None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:27.017813 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d705f7a-2bac-4542-8b94-38e486808afe None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:27.021113 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d705f7a-2bac-4542-8b94-38e486808afe None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:27.021113 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 245/471] 10.222.0.30 () {54 vars in 927 bytes} [Tue Sep 6 04:17:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 04:17:27.024887 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d705f7a-2bac-4542-8b94-38e486808afe None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:27.115463 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311834f4-4ae5-43ee-aae7-5e613aa53e0a None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:27.152805 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e92ea73-4b02-4597-bd4e-7227c86fb7e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:27.153477 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e92ea73-4b02-4597-bd4e-7227c86fb7e8 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:27.153757 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e92ea73-4b02-4597-bd4e-7227c86fb7e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:27.154027 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e92ea73-4b02-4597-bd4e-7227c86fb7e8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:27.154319 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d705f7a-2bac-4542-8b94-38e486808afe None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:27.167335 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:17:27.168601 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e92ea73-4b02-4597-bd4e-7227c86fb7e8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:17:27.170142 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e92ea73-4b02-4597-bd4e-7227c86fb7e8 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:17:27.185802 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-5e92ea73-4b02-4597-bd4e-7227c86fb7e8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 04:17:27.185802 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 04:17:27.185802 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 04:17:27.185802 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 04:17:27.185802 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 04:17:27.185802 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 04:17:27.185802 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 04:17:27.185802 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 04:17:27.185802 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 04:17:27.185802 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 04:17:27.185802 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 04:17:27.185802 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 04:17:27.185802 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 04:17:27.185802 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 04:17:27.185802 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 04:17:27.185802 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 04:17:27.186754 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 04:17:27.186754 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 04:17:27.186754 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 04:17:27.186754 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 04:17:27.186754 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 04:17:27.186754 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 04:17:27.186754 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 04:17:27.186754 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 04:17:27.186754 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 04:17:27.186754 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 04:17:27.186754 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 04:17:27.186754 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 04:17:27.186754 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 04:17:27.186754 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 04:17:27.186754 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 04:17:27.186754 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 04:17:27.186754 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 04:17:27.187607 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 04:17:27.187607 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 04:17:27.187607 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 04:17:27.187607 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 227/472] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Sep 6 04:17:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 04:17:27.188403 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773cf74a-23fe-4833-884d-669f2ff4d9de None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:27.188685 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773cf74a-23fe-4833-884d-669f2ff4d9de None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:27.188958 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773cf74a-23fe-4833-884d-669f2ff4d9de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:27.204127 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66bc9678-fa43-4e63-9aef-bdc50071249c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:27.205884 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bc9678-fa43-4e63-9aef-bdc50071249c None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:27.206384 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bc9678-fa43-4e63-9aef-bdc50071249c None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:27.206657 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bc9678-fa43-4e63-9aef-bdc50071249c None admin] PATH_INFO: `/v3/domains` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:27.207867 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66bc9678-fa43-4e63-9aef-bdc50071249c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:17:27.208338 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66bc9678-fa43-4e63-9aef-bdc50071249c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:17:27.209573 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-773cf74a-23fe-4833-884d-669f2ff4d9de None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:17:27.209934 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66bc9678-fa43-4e63-9aef-bdc50071249c None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:17:27.213155 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-773cf74a-23fe-4833-884d-669f2ff4d9de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:17:27.227546 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 246/473] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Sep 6 04:17:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 04:17:27.233199 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66bc9678-fa43-4e63-9aef-bdc50071249c None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:27.283603 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:17:27.294682 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 228/474] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Sep 6 04:17:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 04:17:27.295438 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773cf74a-23fe-4833-884d-669f2ff4d9de None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:27.371452 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66bc9678-fa43-4e63-9aef-bdc50071249c None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:27.425511 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c172f4be-5d21-4cd0-b76d-64ab664cf40f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:27.426483 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c172f4be-5d21-4cd0-b76d-64ab664cf40f None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:27.426885 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c172f4be-5d21-4cd0-b76d-64ab664cf40f None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:27.427259 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c172f4be-5d21-4cd0-b76d-64ab664cf40f None admin] PATH_INFO: `/v3/domains` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:27.428015 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c172f4be-5d21-4cd0-b76d-64ab664cf40f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:17:27.428421 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c172f4be-5d21-4cd0-b76d-64ab664cf40f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:17:27.428860 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773cf74a-23fe-4833-884d-669f2ff4d9de None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:27.430113 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c172f4be-5d21-4cd0-b76d-64ab664cf40f None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:17:27.446522 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 247/475] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Sep 6 04:17:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:17:27.452101 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c172f4be-5d21-4cd0-b76d-64ab664cf40f None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:27.478065 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:17:27.500621 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36688d91-2ad9-4097-a176-e09f75c49f0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:27.501380 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36688d91-2ad9-4097-a176-e09f75c49f0a None admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:27.501560 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36688d91-2ad9-4097-a176-e09f75c49f0a None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:27.501835 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36688d91-2ad9-4097-a176-e09f75c49f0a None admin] PATH_INFO: `/v3/users` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:27.502761 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36688d91-2ad9-4097-a176-e09f75c49f0a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:17:27.505158 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36688d91-2ad9-4097-a176-e09f75c49f0a None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:17:27.594336 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:17:27.598383 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c172f4be-5d21-4cd0-b76d-64ab664cf40f None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:27.623943 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 229/476] 10.222.0.30 () {60 vars in 1132 bytes} [Tue Sep 6 04:17:27 2022] POST /identity/v3/users => generated 248 bytes in 329 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 04:17:27.629487 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36688d91-2ad9-4097-a176-e09f75c49f0a None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:27.662094 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f049d5-312f-4330-9ec5-0765bde63f23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:27.662998 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f049d5-312f-4330-9ec5-0765bde63f23 None admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:27.663595 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f049d5-312f-4330-9ec5-0765bde63f23 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:27.666164 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f049d5-312f-4330-9ec5-0765bde63f23 None admin] PATH_INFO: `/v3/users` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:27.670428 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f049d5-312f-4330-9ec5-0765bde63f23 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:17:27.670871 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f049d5-312f-4330-9ec5-0765bde63f23 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:17:27.775266 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 248/477] 10.222.0.30 () {60 vars in 1132 bytes} [Tue Sep 6 04:17:27 2022] POST /identity/v3/users => generated 249 bytes in 323 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 04:17:27.789428 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36688d91-2ad9-4097-a176-e09f75c49f0a None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:27.831242 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d83b37-ea52-4591-adb4-cc45910eda65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:27.831877 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d83b37-ea52-4591-adb4-cc45910eda65 None admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:27.832217 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d83b37-ea52-4591-adb4-cc45910eda65 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:27.832549 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d83b37-ea52-4591-adb4-cc45910eda65 None admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:27.833239 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d83b37-ea52-4591-adb4-cc45910eda65 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:17:27.834690 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d83b37-ea52-4591-adb4-cc45910eda65 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:17:27.865822 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.sql.core [None req-32d83b37-ea52-4591-adb4-cc45910eda65 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 06 04:17:27.865822 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Sep 06 04:17:27.865822 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [parameters: {'id': '4d0c987c759e4e0584bcc8a5e6a73f6e', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Sep 06 04:17:27.865822 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81697) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 06 04:17:27.887121 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-32d83b37-ea52-4591-adb4-cc45910eda65 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Sep 06 04:17:27.887121 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 04:17:27.887121 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 06 04:17:27.887121 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 06 04:17:27.887121 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 06 04:17:27.887121 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 06 04:17:27.887121 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 06 04:17:27.887121 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application result = self._query(query) Sep 06 04:17:27.887121 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 06 04:17:27.887121 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application conn.query(q) Sep 06 04:17:27.887121 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 06 04:17:27.887121 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 06 04:17:27.887121 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 06 04:17:27.887121 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application result.read() Sep 06 04:17:27.887121 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 06 04:17:27.888186 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 06 04:17:27.888186 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 06 04:17:27.888186 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 06 04:17:27.888186 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 06 04:17:27.888186 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 06 04:17:27.888186 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 06 04:17:27.888186 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 06 04:17:27.888186 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 06 04:17:27.888186 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 04:17:27.888186 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 06 04:17:27.888186 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 04:17:27.888186 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 04:17:27.888186 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 06 04:17:27.888186 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 06 04:17:27.888186 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Sep 06 04:17:27.888186 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return ref.to_dict() Sep 06 04:17:27.888186 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 06 04:17:27.888186 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application next(self.gen) Sep 06 04:17:27.889126 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 06 04:17:27.889126 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application yield resource Sep 06 04:17:27.889126 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 06 04:17:27.889126 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application next(self.gen) Sep 06 04:17:27.889126 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 06 04:17:27.889126 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application self.session.rollback() Sep 06 04:17:27.889126 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 06 04:17:27.889126 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application self.force_reraise() Sep 06 04:17:27.889126 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 06 04:17:27.889126 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise self.value Sep 06 04:17:27.889126 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 06 04:17:27.889126 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 06 04:17:27.889126 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 06 04:17:27.889126 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application session.commit() Sep 06 04:17:27.889126 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 06 04:17:27.889126 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 06 04:17:27.889126 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 06 04:17:27.890076 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application self._prepare_impl() Sep 06 04:17:27.890076 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 06 04:17:27.890076 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application self.session.flush() Sep 06 04:17:27.890076 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 06 04:17:27.890076 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application self._flush(objects) Sep 06 04:17:27.890076 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 06 04:17:27.890076 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 06 04:17:27.890076 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 06 04:17:27.890076 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application compat.raise_( Sep 06 04:17:27.890076 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 06 04:17:27.890076 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception Sep 06 04:17:27.890076 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 06 04:17:27.890076 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application flush_context.execute() Sep 06 04:17:27.890076 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 06 04:17:27.890076 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rec.execute(self) Sep 06 04:17:27.890076 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 06 04:17:27.890076 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 06 04:17:27.891007 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 06 04:17:27.891007 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 06 04:17:27.891007 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Sep 06 04:17:27.891007 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application c = connection._execute_20( Sep 06 04:17:27.891007 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 06 04:17:27.891007 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 06 04:17:27.891007 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 06 04:17:27.891007 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 06 04:17:27.891007 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 06 04:17:27.891007 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 06 04:17:27.891007 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 06 04:17:27.891007 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 06 04:17:27.891007 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 06 04:17:27.891007 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 06 04:17:27.891007 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 06 04:17:27.891007 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception Sep 06 04:17:27.891846 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 06 04:17:27.891846 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 06 04:17:27.891846 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 06 04:17:27.891846 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 06 04:17:27.891846 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 06 04:17:27.891846 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application result = self._query(query) Sep 06 04:17:27.891846 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 06 04:17:27.891846 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application conn.query(q) Sep 06 04:17:27.891846 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 06 04:17:27.891846 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 06 04:17:27.891846 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 06 04:17:27.891846 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application result.read() Sep 06 04:17:27.891846 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 06 04:17:27.891846 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 06 04:17:27.891846 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 06 04:17:27.891846 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 06 04:17:27.892770 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 06 04:17:27.892770 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 06 04:17:27.892770 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 06 04:17:27.892770 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 06 04:17:27.892770 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 06 04:17:27.892770 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Sep 06 04:17:27.892770 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [parameters: {'id': '4d0c987c759e4e0584bcc8a5e6a73f6e', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Sep 06 04:17:27.892770 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 06 04:17:27.892770 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 04:17:27.892770 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 06 04:17:27.892770 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 04:17:27.892770 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 04:17:27.892770 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 04:17:27.892770 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 04:17:27.892770 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 04:17:27.892770 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 04:17:27.895276 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 04:17:27.895276 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 04:17:27.895276 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 04:17:27.895276 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 04:17:27.895276 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 04:17:27.895276 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 04:17:27.895276 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Sep 06 04:17:27.895276 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Sep 06 04:17:27.895276 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 04:17:27.895276 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 04:17:27.895276 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Sep 06 04:17:27.895276 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Sep 06 04:17:27.895276 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 06 04:17:27.895276 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 06 04:17:27.895276 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Sep 06 04:17:27.895276 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 04:17:27.897238 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 230/478] 10.222.0.30 () {60 vars in 1131 bytes} [Tue Sep 6 04:17:27 2022] POST /identity/v3/roles => generated 144 bytes in 264 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Sep 06 04:17:27.899955 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f049d5-312f-4330-9ec5-0765bde63f23 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:28.027970 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f049d5-312f-4330-9ec5-0765bde63f23 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:28.070248 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1add85ee-adee-4142-8538-28aff2407ab6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:28.071117 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1add85ee-adee-4142-8538-28aff2407ab6 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:28.071559 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1add85ee-adee-4142-8538-28aff2407ab6 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:28.071966 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1add85ee-adee-4142-8538-28aff2407ab6 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:28.087492 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1add85ee-adee-4142-8538-28aff2407ab6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 04:17:28.092946 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-1add85ee-adee-4142-8538-28aff2407ab6 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 06 04:17:28.092946 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 04:17:28.092946 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 04:17:28.092946 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 04:17:28.092946 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 04:17:28.092946 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 04:17:28.092946 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 04:17:28.092946 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 04:17:28.092946 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 04:17:28.092946 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 04:17:28.092946 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 04:17:28.092946 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 04:17:28.092946 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 04:17:28.092946 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 04:17:28.092946 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 04:17:28.092946 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 04:17:28.094081 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 04:17:28.094081 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 04:17:28.094081 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 04:17:28.094081 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 04:17:28.094081 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 04:17:28.094081 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 04:17:28.094081 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 04:17:28.094081 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 04:17:28.094081 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 04:17:28.094081 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 04:17:28.094081 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 04:17:28.094081 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 04:17:28.094081 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 04:17:28.094081 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 04:17:28.094081 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 04:17:28.094081 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 04:17:28.095082 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 04:17:28.095082 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 04:17:28.095082 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 04:17:28.095082 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 04:17:28.095082 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 04:17:28.095082 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 04:17:28.095082 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 04:17:28.095082 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 04:17:28.095082 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 04:17:28.095082 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 04:17:28.095082 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 06 04:17:28.095082 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 04:17:28.101400 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 249/479] 10.222.0.30 () {56 vars in 1114 bytes} [Tue Sep 6 04:17:27 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 04:17:28.103693 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d83b37-ea52-4591-adb4-cc45910eda65 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:28.154217 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:17:28.216718 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d83b37-ea52-4591-adb4-cc45910eda65 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:28.253130 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2c13e6-f394-4302-b9bd-9a9abd4d34a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:28.254678 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2c13e6-f394-4302-b9bd-9a9abd4d34a7 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:28.255152 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2c13e6-f394-4302-b9bd-9a9abd4d34a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:28.255559 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2c13e6-f394-4302-b9bd-9a9abd4d34a7 None admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:28.256367 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2c13e6-f394-4302-b9bd-9a9abd4d34a7 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:17:28.256806 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2c13e6-f394-4302-b9bd-9a9abd4d34a7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:17:28.258375 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2c13e6-f394-4302-b9bd-9a9abd4d34a7 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:17:28.269477 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 231/480] 10.222.0.30 () {56 vars in 1111 bytes} [Tue Sep 6 04:17:28 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:17:29.095090 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4a4927-ea4a-4466-acc7-a3fb77631dfb None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:29.096099 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4a4927-ea4a-4466-acc7-a3fb77631dfb None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:29.096580 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4a4927-ea4a-4466-acc7-a3fb77631dfb None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:29.098380 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 250/481] 10.222.0.30 () {54 vars in 927 bytes} [Tue Sep 6 04:17:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 04:17:29.110937 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b266fc-79c8-4994-85f2-5618b64bb733 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:29.111818 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b266fc-79c8-4994-85f2-5618b64bb733 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:29.112287 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b266fc-79c8-4994-85f2-5618b64bb733 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:29.133117 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-39b266fc-79c8-4994-85f2-5618b64bb733 None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:17:29.136293 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-39b266fc-79c8-4994-85f2-5618b64bb733 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:17:29.216392 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 232/482] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Sep 6 04:17:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 04:17:29.276039 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f8430f-5b02-47ca-8381-8ad68942aa74 None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:29.276473 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f8430f-5b02-47ca-8381-8ad68942aa74 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:29.276854 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f8430f-5b02-47ca-8381-8ad68942aa74 None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:29.278585 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 251/483] 10.222.0.30 () {54 vars in 927 bytes} [Tue Sep 6 04:17:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 04:17:29.283991 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756c8830-b2af-4452-b21d-7bcb194b6b93 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:29.284425 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756c8830-b2af-4452-b21d-7bcb194b6b93 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:29.284805 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756c8830-b2af-4452-b21d-7bcb194b6b93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:29.306060 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-756c8830-b2af-4452-b21d-7bcb194b6b93 None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:17:29.307259 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-756c8830-b2af-4452-b21d-7bcb194b6b93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:17:29.348535 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:17:29.400555 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 233/484] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Sep 6 04:17:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 04:17:29.538492 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f8430f-5b02-47ca-8381-8ad68942aa74 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:29.555658 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:17:29.678682 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f8430f-5b02-47ca-8381-8ad68942aa74 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:29.731412 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6608f50c-0377-4405-ba4c-6323349b4422 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:29.732362 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6608f50c-0377-4405-ba4c-6323349b4422 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:29.732817 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6608f50c-0377-4405-ba4c-6323349b4422 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:29.733263 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6608f50c-0377-4405-ba4c-6323349b4422 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:29.750499 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6608f50c-0377-4405-ba4c-6323349b4422 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 04:17:29.755918 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-6608f50c-0377-4405-ba4c-6323349b4422 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 06 04:17:29.755918 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 04:17:29.755918 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 04:17:29.755918 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 04:17:29.755918 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 04:17:29.755918 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 04:17:29.755918 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 04:17:29.755918 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 04:17:29.755918 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 04:17:29.755918 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 04:17:29.755918 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 04:17:29.755918 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 04:17:29.755918 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 04:17:29.755918 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 04:17:29.755918 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 04:17:29.755918 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 04:17:29.756788 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 04:17:29.756788 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 04:17:29.756788 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 04:17:29.756788 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 04:17:29.756788 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 04:17:29.756788 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 04:17:29.756788 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 04:17:29.756788 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 04:17:29.756788 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 04:17:29.756788 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 04:17:29.756788 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 04:17:29.756788 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 04:17:29.756788 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 04:17:29.756788 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 04:17:29.756788 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 04:17:29.756788 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 04:17:29.757646 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 04:17:29.757646 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 04:17:29.757646 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 04:17:29.757646 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 04:17:29.757646 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 04:17:29.757646 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 04:17:29.757646 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 04:17:29.757646 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 04:17:29.757646 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 04:17:29.757646 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 04:17:29.757646 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 06 04:17:29.757646 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 04:17:29.759863 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 252/485] 10.222.0.30 () {56 vars in 1102 bytes} [Tue Sep 6 04:17:29 2022] GET /identity/v3/roles/service => generated 85 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 04:17:29.766416 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756c8830-b2af-4452-b21d-7bcb194b6b93 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:29.815894 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6608f50c-0377-4405-ba4c-6323349b4422 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:29.924885 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:17:29.947880 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6608f50c-0377-4405-ba4c-6323349b4422 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:29.984676 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756c8830-b2af-4452-b21d-7bcb194b6b93 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:29.993126 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f7c267-ef0e-4231-a1ae-670bf3a0b657 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:29.993800 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f7c267-ef0e-4231-a1ae-670bf3a0b657 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:29.994079 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f7c267-ef0e-4231-a1ae-670bf3a0b657 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:29.994345 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f7c267-ef0e-4231-a1ae-670bf3a0b657 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:30.017619 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f7c267-ef0e-4231-a1ae-670bf3a0b657 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 04:17:30.029488 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-90f7c267-ef0e-4231-a1ae-670bf3a0b657 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 06 04:17:30.029488 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 04:17:30.029488 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 04:17:30.029488 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 04:17:30.029488 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 04:17:30.029488 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 04:17:30.029488 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 04:17:30.029488 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 04:17:30.029488 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 04:17:30.029488 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 04:17:30.029488 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 04:17:30.029488 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 04:17:30.029488 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 04:17:30.029488 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 04:17:30.029488 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 04:17:30.029488 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 04:17:30.030405 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 04:17:30.030405 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 04:17:30.030405 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 04:17:30.030405 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 04:17:30.030405 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 04:17:30.030405 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 04:17:30.030405 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 04:17:30.030405 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 04:17:30.030405 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 04:17:30.030405 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 04:17:30.030405 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 04:17:30.030405 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 04:17:30.030405 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 04:17:30.030405 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 04:17:30.030405 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 04:17:30.030405 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 04:17:30.031203 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 04:17:30.031203 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 04:17:30.031203 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 04:17:30.031203 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 04:17:30.031203 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 04:17:30.031203 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 04:17:30.031203 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 04:17:30.031203 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 04:17:30.031203 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 04:17:30.031203 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 04:17:30.031203 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 06 04:17:30.031203 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 04:17:30.031203 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 253/486] 10.222.0.30 () {56 vars in 1102 bytes} [Tue Sep 6 04:17:29 2022] GET /identity/v3/roles/service => generated 85 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 04:17:30.038262 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f7c267-ef0e-4231-a1ae-670bf3a0b657 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:30.052682 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7dd9624-8fac-4842-931b-d10869797f42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:30.053327 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7dd9624-8fac-4842-931b-d10869797f42 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:30.053605 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7dd9624-8fac-4842-931b-d10869797f42 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:30.053883 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7dd9624-8fac-4842-931b-d10869797f42 None admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:30.054556 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7dd9624-8fac-4842-931b-d10869797f42 None admin] RBAC: Adding query filter params (name=service) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:17:30.054857 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7dd9624-8fac-4842-931b-d10869797f42 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:17:30.056227 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7dd9624-8fac-4842-931b-d10869797f42 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:17:30.072883 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 234/487] 10.222.0.30 () {56 vars in 1103 bytes} [Tue Sep 6 04:17:29 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:17:30.078394 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7dd9624-8fac-4842-931b-d10869797f42 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:30.132775 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:17:30.194229 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f7c267-ef0e-4231-a1ae-670bf3a0b657 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:30.231257 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6792fe65-98b2-4768-8b7e-76ad7c9be107 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:30.232234 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6792fe65-98b2-4768-8b7e-76ad7c9be107 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:30.232726 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6792fe65-98b2-4768-8b7e-76ad7c9be107 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:30.233190 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6792fe65-98b2-4768-8b7e-76ad7c9be107 None admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:30.234162 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6792fe65-98b2-4768-8b7e-76ad7c9be107 None admin] RBAC: Adding query filter params (name=service) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:17:30.234663 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6792fe65-98b2-4768-8b7e-76ad7c9be107 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:17:30.236251 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6792fe65-98b2-4768-8b7e-76ad7c9be107 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:17:30.239756 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:17:30.247018 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 254/488] 10.222.0.30 () {56 vars in 1103 bytes} [Tue Sep 6 04:17:30 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 04:17:30.253206 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6792fe65-98b2-4768-8b7e-76ad7c9be107 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:30.302981 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7dd9624-8fac-4842-931b-d10869797f42 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:30.343965 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb6db51-a2b5-41e3-96a8-68d24febfce9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:30.346042 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb6db51-a2b5-41e3-96a8-68d24febfce9 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:30.346325 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb6db51-a2b5-41e3-96a8-68d24febfce9 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:30.346601 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb6db51-a2b5-41e3-96a8-68d24febfce9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:30.358732 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb6db51-a2b5-41e3-96a8-68d24febfce9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:17:30.360194 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb6db51-a2b5-41e3-96a8-68d24febfce9 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:17:30.376016 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-bcb6db51-a2b5-41e3-96a8-68d24febfce9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 04:17:30.376016 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 04:17:30.376016 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 04:17:30.376016 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 04:17:30.376016 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 04:17:30.376016 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 04:17:30.376016 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 04:17:30.376016 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 04:17:30.376016 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 04:17:30.376016 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 04:17:30.376016 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 04:17:30.376016 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 04:17:30.376016 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 04:17:30.376016 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 04:17:30.376016 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 04:17:30.376016 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 04:17:30.376951 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 04:17:30.376951 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 04:17:30.376951 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 04:17:30.376951 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 04:17:30.376951 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 04:17:30.376951 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 04:17:30.376951 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 04:17:30.376951 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 04:17:30.376951 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 04:17:30.376951 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 04:17:30.376951 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 04:17:30.376951 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 04:17:30.376951 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 04:17:30.376951 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 04:17:30.376951 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 04:17:30.376951 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 04:17:30.376951 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 04:17:30.378110 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 04:17:30.378110 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 04:17:30.378110 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 04:17:30.382913 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 235/489] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Sep 6 04:17:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 301 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 04:17:30.384274 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4dd227-1aae-4408-baf9-a90b5511f3c3 None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:30.384575 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4dd227-1aae-4408-baf9-a90b5511f3c3 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:30.385243 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4dd227-1aae-4408-baf9-a90b5511f3c3 None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:30.386293 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 236/490] 10.222.0.30 () {54 vars in 927 bytes} [Tue Sep 6 04:17:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 04:17:30.387176 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021c87c4-4995-4a68-96ec-5df8c1ecc55d None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:30.387462 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021c87c4-4995-4a68-96ec-5df8c1ecc55d None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:30.387734 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021c87c4-4995-4a68-96ec-5df8c1ecc55d None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:30.392922 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 237/491] 10.222.0.30 () {54 vars in 927 bytes} [Tue Sep 6 04:17:30 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 04:17:30.393046 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6792fe65-98b2-4768-8b7e-76ad7c9be107 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:30.393524 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021c87c4-4995-4a68-96ec-5df8c1ecc55d None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:30.427281 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3f85c5-b6e6-4777-a732-2bb466a3d1a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:30.431071 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3f85c5-b6e6-4777-a732-2bb466a3d1a6 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:30.431071 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3f85c5-b6e6-4777-a732-2bb466a3d1a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:30.431071 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3f85c5-b6e6-4777-a732-2bb466a3d1a6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:30.441220 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3f85c5-b6e6-4777-a732-2bb466a3d1a6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:17:30.442798 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3f85c5-b6e6-4777-a732-2bb466a3d1a6 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:17:30.457134 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-eb3f85c5-b6e6-4777-a732-2bb466a3d1a6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 04:17:30.457134 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 04:17:30.457134 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 04:17:30.457134 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 04:17:30.457134 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 04:17:30.457134 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 04:17:30.457134 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 04:17:30.457134 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 04:17:30.457134 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 04:17:30.457134 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 04:17:30.457134 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 04:17:30.457134 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 04:17:30.457134 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 04:17:30.457134 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 04:17:30.457134 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 04:17:30.457134 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 04:17:30.461738 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 04:17:30.461738 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 04:17:30.461738 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 04:17:30.461738 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 04:17:30.461738 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 04:17:30.461738 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 04:17:30.461738 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 04:17:30.461738 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 04:17:30.461738 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 04:17:30.461738 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 04:17:30.461738 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 04:17:30.461738 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 04:17:30.461738 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 04:17:30.461738 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 04:17:30.461738 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 04:17:30.461738 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 04:17:30.461738 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 04:17:30.462606 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 04:17:30.462606 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 04:17:30.462606 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 04:17:30.462606 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 255/492] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Sep 6 04:17:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 04:17:30.462606 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088fd0da-82c1-4963-bf79-923b35e99d33 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:30.462606 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088fd0da-82c1-4963-bf79-923b35e99d33 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:30.462606 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088fd0da-82c1-4963-bf79-923b35e99d33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:30.478474 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-088fd0da-82c1-4963-bf79-923b35e99d33 None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:17:30.479684 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-088fd0da-82c1-4963-bf79-923b35e99d33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:17:30.485150 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:17:30.510380 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021c87c4-4995-4a68-96ec-5df8c1ecc55d None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:30.526229 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:17:30.547090 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739b7393-78a3-406e-b7c3-269fc57477c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:30.547873 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739b7393-78a3-406e-b7c3-269fc57477c0 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:30.548157 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739b7393-78a3-406e-b7c3-269fc57477c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:30.548429 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739b7393-78a3-406e-b7c3-269fc57477c0 None admin] PATH_INFO: `/v3/domains` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:30.549066 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739b7393-78a3-406e-b7c3-269fc57477c0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:17:30.549448 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739b7393-78a3-406e-b7c3-269fc57477c0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:17:30.550950 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739b7393-78a3-406e-b7c3-269fc57477c0 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:17:30.557490 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 256/493] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Sep 6 04:17:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 04:17:30.557490 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193ac689-e23b-4ca5-8cdb-e4e7669a2375 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:30.557490 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193ac689-e23b-4ca5-8cdb-e4e7669a2375 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:30.557490 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193ac689-e23b-4ca5-8cdb-e4e7669a2375 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:30.573253 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 238/494] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Sep 6 04:17:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:17:30.573253 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739b7393-78a3-406e-b7c3-269fc57477c0 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:30.585753 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-193ac689-e23b-4ca5-8cdb-e4e7669a2375 None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:17:30.587909 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-193ac689-e23b-4ca5-8cdb-e4e7669a2375 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:17:30.686132 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 257/495] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Sep 6 04:17:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 04:17:30.687208 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193ac689-e23b-4ca5-8cdb-e4e7669a2375 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:30.692462 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739b7393-78a3-406e-b7c3-269fc57477c0 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:30.759693 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856d4d4b-b8ea-41db-9217-c6ec32102cbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:30.760235 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856d4d4b-b8ea-41db-9217-c6ec32102cbf None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:30.760532 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856d4d4b-b8ea-41db-9217-c6ec32102cbf None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:30.760831 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856d4d4b-b8ea-41db-9217-c6ec32102cbf None admin] PATH_INFO: `/v3/domains` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:30.761569 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856d4d4b-b8ea-41db-9217-c6ec32102cbf None admin] RBAC: Adding query filter params (name=Default) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:17:30.761876 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856d4d4b-b8ea-41db-9217-c6ec32102cbf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:17:30.763259 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856d4d4b-b8ea-41db-9217-c6ec32102cbf None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:17:30.771210 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:17:30.776818 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 239/496] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Sep 6 04:17:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:17:30.782611 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014cc332-cd3c-422b-b7c5-a93e016f79b8 None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:30.783037 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014cc332-cd3c-422b-b7c5-a93e016f79b8 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:30.783366 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014cc332-cd3c-422b-b7c5-a93e016f79b8 None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:30.790717 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 240/497] 10.222.0.30 () {54 vars in 927 bytes} [Tue Sep 6 04:17:30 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 04:17:30.790717 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014cc332-cd3c-422b-b7c5-a93e016f79b8 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:30.807080 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193ac689-e23b-4ca5-8cdb-e4e7669a2375 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:30.819797 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:17:30.849547 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01086508-ace5-4dc1-b6f6-0ab410c50274 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:30.850573 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01086508-ace5-4dc1-b6f6-0ab410c50274 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:30.850857 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01086508-ace5-4dc1-b6f6-0ab410c50274 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:30.851132 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01086508-ace5-4dc1-b6f6-0ab410c50274 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:30.858555 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01086508-ace5-4dc1-b6f6-0ab410c50274 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:17:30.860018 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01086508-ace5-4dc1-b6f6-0ab410c50274 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:17:30.873227 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-01086508-ace5-4dc1-b6f6-0ab410c50274 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 06 04:17:30.873227 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 04:17:30.873227 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 04:17:30.873227 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 04:17:30.873227 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 04:17:30.873227 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 04:17:30.873227 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 04:17:30.873227 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 04:17:30.873227 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 04:17:30.873227 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 04:17:30.873227 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 04:17:30.873227 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 04:17:30.873227 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 06 04:17:30.873227 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 06 04:17:30.873227 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 06 04:17:30.873227 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 06 04:17:30.874270 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 04:17:30.874270 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 04:17:30.874270 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 06 04:17:30.874270 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 04:17:30.874270 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 06 04:17:30.874270 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 04:17:30.874270 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 04:17:30.874270 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 04:17:30.874270 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 04:17:30.874270 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 04:17:30.874270 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 04:17:30.874270 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 04:17:30.874270 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 04:17:30.874270 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 04:17:30.874270 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 04:17:30.874270 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 04:17:30.874270 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 04:17:30.874270 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 04:17:30.875212 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 04:17:30.875212 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 04:17:30.875212 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 06 04:17:30.875212 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 06 04:17:30.875212 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 06 04:17:30.875212 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 06 04:17:30.875212 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 06 04:17:30.875212 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 06 04:17:30.875212 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 06 04:17:30.875212 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 04:17:30.875212 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 258/498] 10.222.0.30 () {56 vars in 1093 bytes} [Tue Sep 6 04:17:30 2022] GET /identity/v3/users/nova => generated 82 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 04:17:30.875812 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c7601c-3711-49b7-bbe6-9c349a4ac0db None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:30.875812 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c7601c-3711-49b7-bbe6-9c349a4ac0db None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:30.876412 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c7601c-3711-49b7-bbe6-9c349a4ac0db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:30.894307 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-e3c7601c-3711-49b7-bbe6-9c349a4ac0db None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:17:30.896102 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-e3c7601c-3711-49b7-bbe6-9c349a4ac0db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:17:30.905855 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014cc332-cd3c-422b-b7c5-a93e016f79b8 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:30.940737 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b35312-eb12-47ee-8049-f5233157a1e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:30.941985 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b35312-eb12-47ee-8049-f5233157a1e5 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:30.942399 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b35312-eb12-47ee-8049-f5233157a1e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:30.942682 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b35312-eb12-47ee-8049-f5233157a1e5 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:30.950434 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b35312-eb12-47ee-8049-f5233157a1e5 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:17:30.952254 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b35312-eb12-47ee-8049-f5233157a1e5 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:17:30.965814 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-f3b35312-eb12-47ee-8049-f5233157a1e5 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 06 04:17:30.965814 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 04:17:30.965814 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 04:17:30.965814 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 04:17:30.965814 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 04:17:30.965814 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 04:17:30.965814 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 04:17:30.965814 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 04:17:30.965814 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 04:17:30.965814 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 04:17:30.965814 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 04:17:30.965814 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 04:17:30.965814 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 06 04:17:30.965814 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 06 04:17:30.965814 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 06 04:17:30.965814 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 06 04:17:30.966813 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 04:17:30.966813 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 04:17:30.966813 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 06 04:17:30.966813 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 04:17:30.966813 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 06 04:17:30.966813 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 04:17:30.966813 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 04:17:30.966813 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 04:17:30.966813 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 04:17:30.966813 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 04:17:30.966813 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 04:17:30.966813 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 04:17:30.966813 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 04:17:30.966813 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 04:17:30.966813 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 04:17:30.966813 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 04:17:30.966813 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 04:17:30.966813 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 04:17:30.967702 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 04:17:30.967702 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 04:17:30.967702 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 06 04:17:30.967702 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 06 04:17:30.967702 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 06 04:17:30.967702 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 06 04:17:30.967702 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 06 04:17:30.967702 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 06 04:17:30.967702 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 06 04:17:30.967702 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 04:17:30.967702 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 241/499] 10.222.0.30 () {56 vars in 1099 bytes} [Tue Sep 6 04:17:30 2022] GET /identity/v3/users/glance => generated 84 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 04:17:30.967702 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b35312-eb12-47ee-8049-f5233157a1e5 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:30.970933 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 259/500] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Sep 6 04:17:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 04:17:30.971772 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c7601c-3711-49b7-bbe6-9c349a4ac0db None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:31.038495 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:17:31.073986 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:17:31.078190 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c7601c-3711-49b7-bbe6-9c349a4ac0db None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:31.087725 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b35312-eb12-47ee-8049-f5233157a1e5 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:31.116274 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553b756c-859c-4f6f-b2e3-e7d4f3d0c448 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:31.116981 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553b756c-859c-4f6f-b2e3-e7d4f3d0c448 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:31.117341 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553b756c-859c-4f6f-b2e3-e7d4f3d0c448 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:31.117625 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553b756c-859c-4f6f-b2e3-e7d4f3d0c448 None admin] PATH_INFO: `/v3/users` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:31.118469 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-553b756c-859c-4f6f-b2e3-e7d4f3d0c448 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:17:31.118779 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-553b756c-859c-4f6f-b2e3-e7d4f3d0c448 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:17:31.120202 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-553b756c-859c-4f6f-b2e3-e7d4f3d0c448 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:17:31.123385 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b60137-76bc-4dcf-945d-901d81560313 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:31.124021 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b60137-76bc-4dcf-945d-901d81560313 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:31.124304 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b60137-76bc-4dcf-945d-901d81560313 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:31.124579 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b60137-76bc-4dcf-945d-901d81560313 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:31.135638 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b60137-76bc-4dcf-945d-901d81560313 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 04:17:31.140915 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-a0b60137-76bc-4dcf-945d-901d81560313 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 06 04:17:31.140915 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 04:17:31.140915 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 04:17:31.140915 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 04:17:31.140915 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 04:17:31.140915 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 04:17:31.140915 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 04:17:31.140915 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 04:17:31.140915 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 04:17:31.140915 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 04:17:31.140915 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 04:17:31.140915 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 04:17:31.140915 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 04:17:31.140915 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 04:17:31.140915 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 04:17:31.140915 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 04:17:31.141979 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 04:17:31.141979 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 04:17:31.141979 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 04:17:31.141979 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 04:17:31.141979 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 04:17:31.141979 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 04:17:31.141979 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 04:17:31.141979 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 04:17:31.141979 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 04:17:31.141979 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 04:17:31.141979 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 04:17:31.141979 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 04:17:31.141979 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 04:17:31.141979 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 04:17:31.141979 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 04:17:31.141979 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 04:17:31.146631 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 04:17:31.146631 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 04:17:31.146631 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 04:17:31.146631 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 04:17:31.146631 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 04:17:31.146631 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 04:17:31.146631 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 04:17:31.146631 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 04:17:31.146631 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 04:17:31.146631 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 04:17:31.146631 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 06 04:17:31.146631 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 04:17:31.146631 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 242/501] 10.222.0.30 () {56 vars in 1102 bytes} [Tue Sep 6 04:17:30 2022] GET /identity/v3/roles/service => generated 85 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 04:17:31.146631 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b60137-76bc-4dcf-945d-901d81560313 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:31.159478 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 260/502] 10.222.0.30 () {56 vars in 1133 bytes} [Tue Sep 6 04:17:30 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:17:31.165039 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553b756c-859c-4f6f-b2e3-e7d4f3d0c448 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:31.251659 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b60137-76bc-4dcf-945d-901d81560313 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:31.264279 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553b756c-859c-4f6f-b2e3-e7d4f3d0c448 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:31.269415 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:17:31.286586 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd95b564-0580-403b-99d9-264efb921832 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:31.287248 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd95b564-0580-403b-99d9-264efb921832 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:31.287531 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd95b564-0580-403b-99d9-264efb921832 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:31.287798 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd95b564-0580-403b-99d9-264efb921832 None admin] PATH_INFO: `/v3/users` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:31.288566 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd95b564-0580-403b-99d9-264efb921832 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:17:31.288821 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd95b564-0580-403b-99d9-264efb921832 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:17:31.290308 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd95b564-0580-403b-99d9-264efb921832 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:17:31.297716 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c20bec-22d4-4633-8d44-6582d4292238 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:31.298357 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c20bec-22d4-4633-8d44-6582d4292238 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:31.298633 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c20bec-22d4-4633-8d44-6582d4292238 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:31.298897 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c20bec-22d4-4633-8d44-6582d4292238 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:31.309508 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c20bec-22d4-4633-8d44-6582d4292238 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 04:17:31.314394 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-92c20bec-22d4-4633-8d44-6582d4292238 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 06 04:17:31.314394 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 04:17:31.314394 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 04:17:31.314394 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 04:17:31.314394 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 04:17:31.314394 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 04:17:31.314394 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 04:17:31.314394 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 04:17:31.314394 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 04:17:31.314394 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 04:17:31.314394 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 04:17:31.314394 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 04:17:31.314394 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 04:17:31.314394 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 04:17:31.314394 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 04:17:31.314394 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 04:17:31.315254 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 04:17:31.315254 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 04:17:31.315254 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 04:17:31.315254 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 04:17:31.315254 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 04:17:31.315254 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 04:17:31.315254 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 04:17:31.315254 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 04:17:31.315254 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 04:17:31.315254 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 04:17:31.315254 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 04:17:31.315254 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 04:17:31.315254 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 04:17:31.315254 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 04:17:31.315254 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 04:17:31.315254 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 04:17:31.317959 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 04:17:31.317959 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 04:17:31.317959 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 04:17:31.317959 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 04:17:31.317959 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 04:17:31.317959 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 04:17:31.317959 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 04:17:31.317959 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 04:17:31.317959 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 04:17:31.317959 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 04:17:31.317959 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 06 04:17:31.317959 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 04:17:31.317959 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 261/503] 10.222.0.30 () {56 vars in 1102 bytes} [Tue Sep 6 04:17:31 2022] GET /identity/v3/roles/service => generated 85 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 04:17:31.317959 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c20bec-22d4-4633-8d44-6582d4292238 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:31.328216 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 243/504] 10.222.0.30 () {56 vars in 1137 bytes} [Tue Sep 6 04:17:31 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:17:31.329047 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd95b564-0580-403b-99d9-264efb921832 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:31.338758 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:17:31.424043 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c20bec-22d4-4633-8d44-6582d4292238 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:31.429895 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd95b564-0580-403b-99d9-264efb921832 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:31.458018 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34efb54b-75b9-4a82-ae18-d9a84ecbf519 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:31.458643 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34efb54b-75b9-4a82-ae18-d9a84ecbf519 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:31.458918 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34efb54b-75b9-4a82-ae18-d9a84ecbf519 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:31.459186 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34efb54b-75b9-4a82-ae18-d9a84ecbf519 None admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:31.459848 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34efb54b-75b9-4a82-ae18-d9a84ecbf519 None admin] RBAC: Adding query filter params (name=service) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:17:31.460144 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34efb54b-75b9-4a82-ae18-d9a84ecbf519 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:17:31.461551 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34efb54b-75b9-4a82-ae18-d9a84ecbf519 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:17:31.465434 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1edef125-4b11-4eb9-836f-f33a1d34f111 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:31.466047 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edef125-4b11-4eb9-836f-f33a1d34f111 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:31.466787 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edef125-4b11-4eb9-836f-f33a1d34f111 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:31.467028 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edef125-4b11-4eb9-836f-f33a1d34f111 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:31.469487 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 262/505] 10.222.0.30 () {56 vars in 1103 bytes} [Tue Sep 6 04:17:31 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:17:31.470050 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34efb54b-75b9-4a82-ae18-d9a84ecbf519 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:31.516513 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:17:31.539829 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1edef125-4b11-4eb9-836f-f33a1d34f111 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:17:31.541277 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1edef125-4b11-4eb9-836f-f33a1d34f111 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:17:31.545688 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:17:31.566445 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34efb54b-75b9-4a82-ae18-d9a84ecbf519 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:31.599418 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c752123f-6a4b-4cb7-9a59-db4e88b658dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:31.600034 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c752123f-6a4b-4cb7-9a59-db4e88b658dd None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:31.600313 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c752123f-6a4b-4cb7-9a59-db4e88b658dd None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:31.600581 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c752123f-6a4b-4cb7-9a59-db4e88b658dd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:31.610766 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c752123f-6a4b-4cb7-9a59-db4e88b658dd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:17:31.612241 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c752123f-6a4b-4cb7-9a59-db4e88b658dd None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:17:31.626576 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-c752123f-6a4b-4cb7-9a59-db4e88b658dd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 04:17:31.626576 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 04:17:31.626576 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 04:17:31.626576 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 04:17:31.626576 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 04:17:31.626576 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 04:17:31.626576 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 04:17:31.626576 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 04:17:31.626576 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 04:17:31.626576 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 04:17:31.626576 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 04:17:31.626576 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 04:17:31.626576 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 04:17:31.626576 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 04:17:31.626576 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 04:17:31.626576 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 04:17:31.627962 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 04:17:31.627962 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 04:17:31.627962 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 04:17:31.627962 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 04:17:31.627962 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 04:17:31.627962 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 04:17:31.627962 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 04:17:31.627962 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 04:17:31.627962 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 04:17:31.627962 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 04:17:31.627962 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 04:17:31.627962 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 04:17:31.627962 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 04:17:31.627962 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 04:17:31.627962 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 04:17:31.627962 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 04:17:31.627962 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 04:17:31.629617 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 04:17:31.629617 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 04:17:31.629617 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 04:17:31.629617 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 263/506] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Sep 6 04:17:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 04:17:31.629617 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c752123f-6a4b-4cb7-9a59-db4e88b658dd None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:31.641523 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 244/507] 10.222.0.30 () {58 vars in 1282 bytes} [Tue Sep 6 04:17:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 06 04:17:31.642348 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1edef125-4b11-4eb9-836f-f33a1d34f111 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:31.724116 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:17:31.736727 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c752123f-6a4b-4cb7-9a59-db4e88b658dd None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:31.747511 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1edef125-4b11-4eb9-836f-f33a1d34f111 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:31.765885 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:17:31.771517 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4443ebf4-0923-404e-8fa4-1ceb9a5c07f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:31.772122 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4443ebf4-0923-404e-8fa4-1ceb9a5c07f2 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:31.772396 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4443ebf4-0923-404e-8fa4-1ceb9a5c07f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:31.772663 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4443ebf4-0923-404e-8fa4-1ceb9a5c07f2 None admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:31.773331 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4443ebf4-0923-404e-8fa4-1ceb9a5c07f2 None admin] RBAC: Adding query filter params (name=service) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:17:31.778399 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4443ebf4-0923-404e-8fa4-1ceb9a5c07f2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:17:31.778399 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4443ebf4-0923-404e-8fa4-1ceb9a5c07f2 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:17:31.780926 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36685bb-f05c-4421-b023-33759a679f14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:31.781568 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36685bb-f05c-4421-b023-33759a679f14 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:31.781848 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36685bb-f05c-4421-b023-33759a679f14 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:31.782121 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36685bb-f05c-4421-b023-33759a679f14 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:31.783061 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 264/508] 10.222.0.30 () {56 vars in 1103 bytes} [Tue Sep 6 04:17:31 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:17:31.783831 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4443ebf4-0923-404e-8fa4-1ceb9a5c07f2 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:31.850112 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36685bb-f05c-4421-b023-33759a679f14 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:17:31.851552 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36685bb-f05c-4421-b023-33759a679f14 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:17:31.884590 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4443ebf4-0923-404e-8fa4-1ceb9a5c07f2 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:31.918559 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a85d28-417e-4f96-9a86-c623b05e3068 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:31.919184 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a85d28-417e-4f96-9a86-c623b05e3068 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:31.919467 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a85d28-417e-4f96-9a86-c623b05e3068 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:31.919737 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a85d28-417e-4f96-9a86-c623b05e3068 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:31.927093 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:17:31.929805 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a85d28-417e-4f96-9a86-c623b05e3068 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:17:31.931272 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a85d28-417e-4f96-9a86-c623b05e3068 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:17:31.946557 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-39a85d28-417e-4f96-9a86-c623b05e3068 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 04:17:31.946557 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 04:17:31.946557 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 04:17:31.946557 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 04:17:31.946557 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 04:17:31.946557 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 04:17:31.946557 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 04:17:31.946557 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 04:17:31.946557 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 04:17:31.946557 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 04:17:31.946557 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 04:17:31.946557 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 04:17:31.946557 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 04:17:31.946557 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 04:17:31.946557 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 04:17:31.946557 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 04:17:31.947448 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 04:17:31.947448 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 04:17:31.947448 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 04:17:31.947448 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 04:17:31.947448 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 04:17:31.947448 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 04:17:31.947448 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 04:17:31.947448 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 04:17:31.947448 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 04:17:31.947448 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 04:17:31.947448 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 04:17:31.947448 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 04:17:31.947448 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 04:17:31.947448 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 04:17:31.947448 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 04:17:31.947448 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 04:17:31.947448 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 04:17:31.948276 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 04:17:31.948276 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 04:17:31.948276 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 04:17:31.948276 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 265/509] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Sep 6 04:17:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 04:17:31.948487 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a85d28-417e-4f96-9a86-c623b05e3068 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:31.955148 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 245/510] 10.222.0.30 () {58 vars in 1282 bytes} [Tue Sep 6 04:17:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 06 04:17:31.955981 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36685bb-f05c-4421-b023-33759a679f14 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:31.994328 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:17:32.048765 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a85d28-417e-4f96-9a86-c623b05e3068 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:32.055862 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36685bb-f05c-4421-b023-33759a679f14 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:32.081190 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b53cbe-f0d2-4931-b6f8-820a1b7ad34f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:32.082403 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b53cbe-f0d2-4931-b6f8-820a1b7ad34f None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:32.082683 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b53cbe-f0d2-4931-b6f8-820a1b7ad34f None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:32.082948 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b53cbe-f0d2-4931-b6f8-820a1b7ad34f None admin] PATH_INFO: `/v3/domains` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:32.083576 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b53cbe-f0d2-4931-b6f8-820a1b7ad34f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:17:32.083868 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b53cbe-f0d2-4931-b6f8-820a1b7ad34f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:17:32.085227 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b53cbe-f0d2-4931-b6f8-820a1b7ad34f None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:17:32.089263 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc574fbb-4438-481e-b371-0bbe9ea2d77c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:32.089919 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc574fbb-4438-481e-b371-0bbe9ea2d77c None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:32.090201 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc574fbb-4438-481e-b371-0bbe9ea2d77c None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:32.090476 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc574fbb-4438-481e-b371-0bbe9ea2d77c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:32.096445 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 266/511] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Sep 6 04:17:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:17:32.097218 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b53cbe-f0d2-4931-b6f8-820a1b7ad34f None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:32.105814 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc574fbb-4438-481e-b371-0bbe9ea2d77c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:17:32.107216 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc574fbb-4438-481e-b371-0bbe9ea2d77c None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:17:32.122972 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-dc574fbb-4438-481e-b371-0bbe9ea2d77c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 04:17:32.122972 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 04:17:32.122972 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 04:17:32.122972 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 04:17:32.122972 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 04:17:32.122972 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 04:17:32.122972 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 04:17:32.122972 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 04:17:32.122972 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 04:17:32.122972 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 04:17:32.122972 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 04:17:32.122972 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 04:17:32.122972 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 04:17:32.122972 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 04:17:32.122972 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 04:17:32.122972 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 04:17:32.123915 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 04:17:32.123915 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 04:17:32.123915 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 04:17:32.123915 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 04:17:32.123915 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 04:17:32.123915 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 04:17:32.123915 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 04:17:32.123915 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 04:17:32.123915 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 04:17:32.123915 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 04:17:32.123915 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 04:17:32.123915 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 04:17:32.123915 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 04:17:32.123915 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 04:17:32.123915 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 04:17:32.123915 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 04:17:32.123915 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 04:17:32.124747 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 04:17:32.124747 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 04:17:32.124747 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 04:17:32.124747 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 246/512] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Sep 6 04:17:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 04:17:32.124969 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc574fbb-4438-481e-b371-0bbe9ea2d77c None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:32.152926 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:17:32.201026 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b53cbe-f0d2-4931-b6f8-820a1b7ad34f None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:32.203295 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:17:32.224913 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc574fbb-4438-481e-b371-0bbe9ea2d77c None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:32.235727 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1383d7b8-fb1b-4964-b694-5f73051d0a9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:32.236365 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1383d7b8-fb1b-4964-b694-5f73051d0a9a None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:32.236643 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1383d7b8-fb1b-4964-b694-5f73051d0a9a None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:32.236914 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1383d7b8-fb1b-4964-b694-5f73051d0a9a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:32.247003 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1383d7b8-fb1b-4964-b694-5f73051d0a9a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:17:32.248461 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1383d7b8-fb1b-4964-b694-5f73051d0a9a None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:17:32.259010 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e0cc66-821b-4ccb-a60d-039dc4b88686 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:32.259618 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e0cc66-821b-4ccb-a60d-039dc4b88686 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:32.259896 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e0cc66-821b-4ccb-a60d-039dc4b88686 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:32.260164 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e0cc66-821b-4ccb-a60d-039dc4b88686 None admin] PATH_INFO: `/v3/domains` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:32.260799 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e0cc66-821b-4ccb-a60d-039dc4b88686 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:17:32.261098 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e0cc66-821b-4ccb-a60d-039dc4b88686 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:17:32.262493 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e0cc66-821b-4ccb-a60d-039dc4b88686 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:17:32.263053 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-1383d7b8-fb1b-4964-b694-5f73051d0a9a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 04:17:32.263053 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 04:17:32.263053 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 04:17:32.263053 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 04:17:32.263053 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 04:17:32.263053 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 04:17:32.263053 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 04:17:32.263053 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 04:17:32.263053 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 04:17:32.263053 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 04:17:32.263053 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 04:17:32.263053 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 04:17:32.263053 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 04:17:32.263053 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 04:17:32.263053 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 04:17:32.263053 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 04:17:32.264006 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 04:17:32.264006 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 04:17:32.264006 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 04:17:32.264006 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 04:17:32.264006 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 04:17:32.264006 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 04:17:32.264006 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 04:17:32.264006 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 04:17:32.264006 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 04:17:32.264006 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 04:17:32.264006 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 04:17:32.264006 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 04:17:32.264006 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 04:17:32.264006 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 04:17:32.264006 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 04:17:32.264006 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 04:17:32.264006 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 04:17:32.265358 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 04:17:32.265358 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 04:17:32.265358 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 04:17:32.265358 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 267/513] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Sep 6 04:17:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 04:17:32.265358 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1383d7b8-fb1b-4964-b694-5f73051d0a9a None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:32.283445 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 247/514] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Sep 6 04:17:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:17:32.283445 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e0cc66-821b-4ccb-a60d-039dc4b88686 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:32.368064 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:17:32.369531 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1383d7b8-fb1b-4964-b694-5f73051d0a9a None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:32.379965 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e0cc66-821b-4ccb-a60d-039dc4b88686 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:32.405544 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38255af-b4bf-4f6e-aad1-586eb1ec2a67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:32.406166 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38255af-b4bf-4f6e-aad1-586eb1ec2a67 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:32.406439 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38255af-b4bf-4f6e-aad1-586eb1ec2a67 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:32.406712 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38255af-b4bf-4f6e-aad1-586eb1ec2a67 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:32.414964 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93a107d-01ac-4cf7-b94d-e3b6dab1b039 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:32.415573 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93a107d-01ac-4cf7-b94d-e3b6dab1b039 None admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:32.415861 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93a107d-01ac-4cf7-b94d-e3b6dab1b039 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:32.416137 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93a107d-01ac-4cf7-b94d-e3b6dab1b039 None admin] PATH_INFO: `/v3/users` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:32.416787 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f93a107d-01ac-4cf7-b94d-e3b6dab1b039 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:17:32.418107 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f38255af-b4bf-4f6e-aad1-586eb1ec2a67 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:17:32.418308 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f93a107d-01ac-4cf7-b94d-e3b6dab1b039 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:17:32.419560 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f38255af-b4bf-4f6e-aad1-586eb1ec2a67 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:17:32.433774 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-f38255af-b4bf-4f6e-aad1-586eb1ec2a67 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 04:17:32.433774 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 04:17:32.433774 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 04:17:32.433774 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 04:17:32.433774 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 04:17:32.433774 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 04:17:32.433774 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 04:17:32.433774 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 04:17:32.433774 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 04:17:32.433774 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 04:17:32.433774 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 04:17:32.433774 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 04:17:32.433774 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 04:17:32.433774 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 04:17:32.433774 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 04:17:32.433774 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 04:17:32.434829 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 04:17:32.434829 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 04:17:32.434829 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 04:17:32.434829 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 04:17:32.434829 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 04:17:32.434829 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 04:17:32.434829 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 04:17:32.434829 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 04:17:32.434829 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 04:17:32.434829 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 04:17:32.434829 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 04:17:32.434829 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 04:17:32.434829 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 04:17:32.434829 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 04:17:32.434829 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 04:17:32.434829 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 04:17:32.434829 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 04:17:32.435730 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 04:17:32.435730 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 04:17:32.435730 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 04:17:32.435730 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 268/515] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Sep 6 04:17:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 04:17:32.435730 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38255af-b4bf-4f6e-aad1-586eb1ec2a67 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:32.446828 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:17:32.498853 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 248/516] 10.222.0.30 () {60 vars in 1132 bytes} [Tue Sep 6 04:17:32 2022] POST /identity/v3/users => generated 247 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 04:17:32.499688 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93a107d-01ac-4cf7-b94d-e3b6dab1b039 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:32.551524 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38255af-b4bf-4f6e-aad1-586eb1ec2a67 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:32.592268 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8989d45-918f-46e6-b952-36d1da3b265d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:32.593258 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8989d45-918f-46e6-b952-36d1da3b265d None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:32.593728 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8989d45-918f-46e6-b952-36d1da3b265d None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:32.594147 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8989d45-918f-46e6-b952-36d1da3b265d None admin] PATH_INFO: `/v3/domains` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:32.594985 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8989d45-918f-46e6-b952-36d1da3b265d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:17:32.595448 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8989d45-918f-46e6-b952-36d1da3b265d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:17:32.597191 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8989d45-918f-46e6-b952-36d1da3b265d None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:17:32.609402 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93a107d-01ac-4cf7-b94d-e3b6dab1b039 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:32.614944 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 269/517] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Sep 6 04:17:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:17:32.616136 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8989d45-918f-46e6-b952-36d1da3b265d None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:32.656866 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27c4a78-2801-48e0-a9ce-b82423d71538 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:32.657447 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27c4a78-2801-48e0-a9ce-b82423d71538 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:32.658641 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27c4a78-2801-48e0-a9ce-b82423d71538 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:32.658641 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27c4a78-2801-48e0-a9ce-b82423d71538 None admin] PATH_INFO: `/v3/domains` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:32.658641 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27c4a78-2801-48e0-a9ce-b82423d71538 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:17:32.658967 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27c4a78-2801-48e0-a9ce-b82423d71538 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:17:32.661690 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27c4a78-2801-48e0-a9ce-b82423d71538 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:17:32.674396 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 249/518] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Sep 6 04:17:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:17:32.679318 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27c4a78-2801-48e0-a9ce-b82423d71538 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:32.683121 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:17:32.701625 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:17:32.738248 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8989d45-918f-46e6-b952-36d1da3b265d None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:32.780550 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1609b5-d5ca-4e7b-b7bc-932bd851409f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:32.781667 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1609b5-d5ca-4e7b-b7bc-932bd851409f None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:32.781950 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1609b5-d5ca-4e7b-b7bc-932bd851409f None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:32.782226 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1609b5-d5ca-4e7b-b7bc-932bd851409f None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:32.790070 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1609b5-d5ca-4e7b-b7bc-932bd851409f None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:17:32.790279 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27c4a78-2801-48e0-a9ce-b82423d71538 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:32.791648 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1609b5-d5ca-4e7b-b7bc-932bd851409f None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:17:32.804546 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-7c1609b5-d5ca-4e7b-b7bc-932bd851409f None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 06 04:17:32.804546 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 04:17:32.804546 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 04:17:32.804546 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 04:17:32.804546 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 04:17:32.804546 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 04:17:32.804546 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 04:17:32.804546 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 04:17:32.804546 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 04:17:32.804546 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 04:17:32.804546 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 04:17:32.804546 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 04:17:32.804546 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 06 04:17:32.804546 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 06 04:17:32.804546 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 06 04:17:32.804546 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 06 04:17:32.805511 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 04:17:32.805511 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 04:17:32.805511 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 06 04:17:32.805511 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 04:17:32.805511 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 06 04:17:32.805511 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 04:17:32.805511 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 04:17:32.805511 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 04:17:32.805511 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 04:17:32.805511 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 04:17:32.805511 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 04:17:32.805511 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 04:17:32.805511 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 04:17:32.805511 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 04:17:32.805511 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 04:17:32.805511 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 04:17:32.805511 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 04:17:32.805511 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 04:17:32.806531 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 04:17:32.806531 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 04:17:32.806531 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 06 04:17:32.806531 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 06 04:17:32.806531 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 06 04:17:32.806531 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 06 04:17:32.806531 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 06 04:17:32.806531 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 06 04:17:32.806531 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 06 04:17:32.806531 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 04:17:32.806531 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 270/519] 10.222.0.30 () {56 vars in 1102 bytes} [Tue Sep 6 04:17:32 2022] GET /identity/v3/users/neutron => generated 85 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 04:17:32.807180 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1609b5-d5ca-4e7b-b7bc-932bd851409f None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:32.828833 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf434188-dc4c-4ba5-9146-5dc4dfbf05ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:32.829474 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf434188-dc4c-4ba5-9146-5dc4dfbf05ca None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:32.829754 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf434188-dc4c-4ba5-9146-5dc4dfbf05ca None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:32.830024 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf434188-dc4c-4ba5-9146-5dc4dfbf05ca None admin] PATH_INFO: `/v3/domains` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:32.830667 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf434188-dc4c-4ba5-9146-5dc4dfbf05ca None admin] RBAC: Adding query filter params (name=Default) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:17:32.830968 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf434188-dc4c-4ba5-9146-5dc4dfbf05ca None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:17:32.832338 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf434188-dc4c-4ba5-9146-5dc4dfbf05ca None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:17:32.843921 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 250/520] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Sep 6 04:17:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:17:32.844722 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf434188-dc4c-4ba5-9146-5dc4dfbf05ca None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:32.917754 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:17:32.918715 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1609b5-d5ca-4e7b-b7bc-932bd851409f None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:32.948531 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:17:32.955958 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5770410c-6591-4978-9d11-b8ff01bd064c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:32.956597 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5770410c-6591-4978-9d11-b8ff01bd064c None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:32.956921 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5770410c-6591-4978-9d11-b8ff01bd064c None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:32.957194 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5770410c-6591-4978-9d11-b8ff01bd064c None admin] PATH_INFO: `/v3/projects/service` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:32.958754 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf434188-dc4c-4ba5-9146-5dc4dfbf05ca None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:32.964088 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5770410c-6591-4978-9d11-b8ff01bd064c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:17:32.965569 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5770410c-6591-4978-9d11-b8ff01bd064c None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:17:32.977789 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-5770410c-6591-4978-9d11-b8ff01bd064c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 04:17:32.977789 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 04:17:32.977789 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 04:17:32.977789 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 04:17:32.977789 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 04:17:32.977789 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 04:17:32.977789 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 04:17:32.977789 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 04:17:32.977789 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 04:17:32.977789 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 04:17:32.977789 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 04:17:32.977789 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 04:17:32.977789 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 06 04:17:32.977789 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 06 04:17:32.977789 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 06 04:17:32.978686 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 06 04:17:32.978686 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 04:17:32.978686 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 04:17:32.978686 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 04:17:32.978686 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 04:17:32.978686 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 04:17:32.978686 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 04:17:32.978686 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 04:17:32.978686 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 04:17:32.978686 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 04:17:32.978686 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 04:17:32.978686 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 04:17:32.978686 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 04:17:32.978686 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 04:17:32.978686 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 04:17:32.978686 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 04:17:32.978686 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 04:17:32.986735 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 06 04:17:32.986735 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 06 04:17:32.986735 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 06 04:17:32.986735 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 06 04:17:32.986735 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 06 04:17:32.986735 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 06 04:17:32.986735 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 04:17:32.986735 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 04:17:32.986735 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 271/521] 10.222.0.30 () {56 vars in 1111 bytes} [Tue Sep 6 04:17:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 04:17:32.986735 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5770410c-6591-4978-9d11-b8ff01bd064c None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:33.006637 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114c17ef-e505-4746-8ddb-337c5f10e6ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:33.007273 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114c17ef-e505-4746-8ddb-337c5f10e6ba None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:33.007553 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114c17ef-e505-4746-8ddb-337c5f10e6ba None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:33.008105 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114c17ef-e505-4746-8ddb-337c5f10e6ba None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:33.016387 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114c17ef-e505-4746-8ddb-337c5f10e6ba None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:17:33.017914 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114c17ef-e505-4746-8ddb-337c5f10e6ba None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:17:33.031341 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-114c17ef-e505-4746-8ddb-337c5f10e6ba None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 06 04:17:33.031341 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 04:17:33.031341 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 04:17:33.031341 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 04:17:33.031341 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 04:17:33.031341 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 04:17:33.031341 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 04:17:33.031341 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 04:17:33.031341 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 04:17:33.031341 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 04:17:33.031341 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 04:17:33.031341 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 04:17:33.031341 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 06 04:17:33.031341 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 06 04:17:33.031341 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 06 04:17:33.031341 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 06 04:17:33.032271 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 04:17:33.032271 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 04:17:33.032271 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 06 04:17:33.032271 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 04:17:33.032271 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 06 04:17:33.032271 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 04:17:33.032271 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 04:17:33.032271 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 04:17:33.032271 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 04:17:33.032271 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 04:17:33.032271 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 04:17:33.032271 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 04:17:33.032271 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 04:17:33.032271 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 04:17:33.032271 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 04:17:33.032271 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 04:17:33.032271 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 04:17:33.032271 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 04:17:33.033197 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 04:17:33.033197 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 04:17:33.033197 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 06 04:17:33.033197 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 06 04:17:33.033197 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 06 04:17:33.033197 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 06 04:17:33.033197 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 06 04:17:33.033197 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 06 04:17:33.033197 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 06 04:17:33.033197 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 04:17:33.033197 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 251/522] 10.222.0.30 () {56 vars in 1099 bytes} [Tue Sep 6 04:17:32 2022] GET /identity/v3/users/cinder => generated 84 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 04:17:33.033197 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114c17ef-e505-4746-8ddb-337c5f10e6ba None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:33.097615 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5770410c-6591-4978-9d11-b8ff01bd064c None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:33.135644 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abdc4a2-9e7c-4cc8-a3f0-45cac0ce1b42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:33.136640 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abdc4a2-9e7c-4cc8-a3f0-45cac0ce1b42 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:33.137203 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abdc4a2-9e7c-4cc8-a3f0-45cac0ce1b42 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:33.137503 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abdc4a2-9e7c-4cc8-a3f0-45cac0ce1b42 None admin] PATH_INFO: `/v3/users` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:33.138384 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3abdc4a2-9e7c-4cc8-a3f0-45cac0ce1b42 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:17:33.138743 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3abdc4a2-9e7c-4cc8-a3f0-45cac0ce1b42 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:17:33.140298 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3abdc4a2-9e7c-4cc8-a3f0-45cac0ce1b42 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:17:33.150014 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114c17ef-e505-4746-8ddb-337c5f10e6ba None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:33.163960 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 272/523] 10.222.0.30 () {56 vars in 1139 bytes} [Tue Sep 6 04:17:32 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:17:33.164848 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abdc4a2-9e7c-4cc8-a3f0-45cac0ce1b42 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:33.172810 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:17:33.184266 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eded79ef-66ed-46be-8f49-07fde3e8f61e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:33.184966 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eded79ef-66ed-46be-8f49-07fde3e8f61e None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:33.185247 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eded79ef-66ed-46be-8f49-07fde3e8f61e None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:33.188239 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eded79ef-66ed-46be-8f49-07fde3e8f61e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:33.192339 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eded79ef-66ed-46be-8f49-07fde3e8f61e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:17:33.193823 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eded79ef-66ed-46be-8f49-07fde3e8f61e None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:17:33.205518 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-eded79ef-66ed-46be-8f49-07fde3e8f61e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 04:17:33.205518 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 04:17:33.205518 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 04:17:33.205518 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 04:17:33.205518 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 04:17:33.205518 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 04:17:33.205518 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 04:17:33.205518 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 04:17:33.205518 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 04:17:33.205518 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 04:17:33.205518 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 04:17:33.205518 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 04:17:33.205518 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 06 04:17:33.205518 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 06 04:17:33.205518 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 06 04:17:33.206398 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 06 04:17:33.206398 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 04:17:33.206398 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 04:17:33.206398 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 04:17:33.206398 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 04:17:33.206398 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 04:17:33.206398 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 04:17:33.206398 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 04:17:33.206398 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 04:17:33.206398 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 04:17:33.206398 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 04:17:33.206398 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 04:17:33.206398 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 04:17:33.206398 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 04:17:33.206398 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 04:17:33.206398 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 04:17:33.206398 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 04:17:33.207332 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 06 04:17:33.207332 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 06 04:17:33.207332 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 06 04:17:33.207332 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 06 04:17:33.207332 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 06 04:17:33.207332 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 06 04:17:33.207332 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 04:17:33.207332 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 04:17:33.211510 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 252/524] 10.222.0.30 () {56 vars in 1111 bytes} [Tue Sep 6 04:17:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 04:17:33.212412 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eded79ef-66ed-46be-8f49-07fde3e8f61e None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:33.215926 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:17:33.274054 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abdc4a2-9e7c-4cc8-a3f0-45cac0ce1b42 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:33.313286 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0eb34fc-e581-409c-b797-6d3c60758bda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:33.313917 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0eb34fc-e581-409c-b797-6d3c60758bda None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:33.314194 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0eb34fc-e581-409c-b797-6d3c60758bda None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:33.314473 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0eb34fc-e581-409c-b797-6d3c60758bda None admin] PATH_INFO: `/v3/projects` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:33.315167 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0eb34fc-e581-409c-b797-6d3c60758bda None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:17:33.315474 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0eb34fc-e581-409c-b797-6d3c60758bda None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:17:33.316942 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0eb34fc-e581-409c-b797-6d3c60758bda None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:17:33.320446 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eded79ef-66ed-46be-8f49-07fde3e8f61e None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:33.340633 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 273/525] 10.222.0.30 () {56 vars in 1148 bytes} [Tue Sep 6 04:17:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:17:33.340633 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0eb34fc-e581-409c-b797-6d3c60758bda None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:33.365435 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719718b9-9e67-4059-825b-47c2edb15718 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:33.366119 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719718b9-9e67-4059-825b-47c2edb15718 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:33.366401 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719718b9-9e67-4059-825b-47c2edb15718 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:33.366676 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719718b9-9e67-4059-825b-47c2edb15718 None admin] PATH_INFO: `/v3/users` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:33.367472 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719718b9-9e67-4059-825b-47c2edb15718 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:17:33.367783 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719718b9-9e67-4059-825b-47c2edb15718 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:17:33.369387 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719718b9-9e67-4059-825b-47c2edb15718 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:17:33.392972 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 253/526] 10.222.0.30 () {56 vars in 1137 bytes} [Tue Sep 6 04:17:33 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:17:33.392972 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719718b9-9e67-4059-825b-47c2edb15718 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:33.436767 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:17:33.459849 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0eb34fc-e581-409c-b797-6d3c60758bda None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:33.485920 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:17:33.497370 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719718b9-9e67-4059-825b-47c2edb15718 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:33.516325 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e86dc5-7268-4709-8758-4762e596b73c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:33.516941 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e86dc5-7268-4709-8758-4762e596b73c None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:33.517216 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e86dc5-7268-4709-8758-4762e596b73c None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:33.517521 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e86dc5-7268-4709-8758-4762e596b73c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:33.533204 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7166f6b8-d1b9-4466-8120-57721d9239a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:33.533892 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7166f6b8-d1b9-4466-8120-57721d9239a1 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:33.534139 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7166f6b8-d1b9-4466-8120-57721d9239a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:33.534413 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7166f6b8-d1b9-4466-8120-57721d9239a1 None admin] PATH_INFO: `/v3/projects` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:33.535142 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7166f6b8-d1b9-4466-8120-57721d9239a1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:17:33.535449 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7166f6b8-d1b9-4466-8120-57721d9239a1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:17:33.536893 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7166f6b8-d1b9-4466-8120-57721d9239a1 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:17:33.563259 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 254/527] 10.222.0.30 () {56 vars in 1148 bytes} [Tue Sep 6 04:17:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:17:33.563259 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7166f6b8-d1b9-4466-8120-57721d9239a1 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:33.596164 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e86dc5-7268-4709-8758-4762e596b73c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:17:33.597721 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e86dc5-7268-4709-8758-4762e596b73c None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:17:33.672957 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7166f6b8-d1b9-4466-8120-57721d9239a1 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:33.674975 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:17:33.704270 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 274/528] 10.222.0.30 () {58 vars in 1282 bytes} [Tue Sep 6 04:17:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 06 04:17:33.705053 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e86dc5-7268-4709-8758-4762e596b73c None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:33.715979 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:17:33.721356 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf0ef7b-d7a2-4c6c-b87e-2583a58f08eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:33.721974 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf0ef7b-d7a2-4c6c-b87e-2583a58f08eb None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:33.722249 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf0ef7b-d7a2-4c6c-b87e-2583a58f08eb None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:33.722554 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf0ef7b-d7a2-4c6c-b87e-2583a58f08eb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:33.723283 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf0ef7b-d7a2-4c6c-b87e-2583a58f08eb None admin] RBAC: Adding query filter params (role.id=83cf0a194f8d45e792411e2d9afdfe50, scope.project.id=b707733ce65c42edbc1fd95ad0a1c06c, user.id=9db138b77afa4b068244b8ef715404fe) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:17:33.723615 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf0ef7b-d7a2-4c6c-b87e-2583a58f08eb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:17:33.725005 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf0ef7b-d7a2-4c6c-b87e-2583a58f08eb None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:17:33.730952 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 255/529] 10.222.0.30 () {56 vars in 1374 bytes} [Tue Sep 6 04:17:33 2022] GET /identity/v3/role_assignments?user.id=9db138b77afa4b068244b8ef715404fe&scope.project.id=b707733ce65c42edbc1fd95ad0a1c06c&role.id=83cf0a194f8d45e792411e2d9afdfe50 => generated 259 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:17:33.742500 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf0ef7b-d7a2-4c6c-b87e-2583a58f08eb None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:33.825462 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e86dc5-7268-4709-8758-4762e596b73c None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:33.868610 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573345f4-e88a-4cae-9e7c-790321d5becd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:33.869615 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573345f4-e88a-4cae-9e7c-790321d5becd None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:33.870635 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573345f4-e88a-4cae-9e7c-790321d5becd None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:33.871108 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573345f4-e88a-4cae-9e7c-790321d5becd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:33.893950 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf0ef7b-d7a2-4c6c-b87e-2583a58f08eb None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:33.936761 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcae0045-8546-4f87-8c71-67463e286d90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:33.937653 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcae0045-8546-4f87-8c71-67463e286d90 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:33.937934 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcae0045-8546-4f87-8c71-67463e286d90 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:33.938627 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcae0045-8546-4f87-8c71-67463e286d90 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:33.939621 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcae0045-8546-4f87-8c71-67463e286d90 None admin] RBAC: Adding query filter params (role.id=83cf0a194f8d45e792411e2d9afdfe50, scope.project.id=b707733ce65c42edbc1fd95ad0a1c06c, user.id=ae5c31fb1d0342c0937bbc83a415113d) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:17:33.940020 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcae0045-8546-4f87-8c71-67463e286d90 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:17:33.942665 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcae0045-8546-4f87-8c71-67463e286d90 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:17:33.953406 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 256/530] 10.222.0.30 () {56 vars in 1374 bytes} [Tue Sep 6 04:17:33 2022] GET /identity/v3/role_assignments?user.id=ae5c31fb1d0342c0937bbc83a415113d&scope.project.id=b707733ce65c42edbc1fd95ad0a1c06c&role.id=83cf0a194f8d45e792411e2d9afdfe50 => generated 259 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:17:33.954398 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcae0045-8546-4f87-8c71-67463e286d90 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:33.957211 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-573345f4-e88a-4cae-9e7c-790321d5becd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:17:33.958944 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-573345f4-e88a-4cae-9e7c-790321d5becd None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:17:33.979569 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:17:34.039208 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:17:34.099533 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 275/531] 10.222.0.30 () {58 vars in 1282 bytes} [Tue Sep 6 04:17:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 06 04:17:34.105446 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573345f4-e88a-4cae-9e7c-790321d5becd None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:34.203492 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcae0045-8546-4f87-8c71-67463e286d90 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:34.240109 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573345f4-e88a-4cae-9e7c-790321d5becd None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:34.248982 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:17:34.268297 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102f8de2-cdda-45c2-bd2a-3958c59a5455 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:34.268951 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102f8de2-cdda-45c2-bd2a-3958c59a5455 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:34.269233 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102f8de2-cdda-45c2-bd2a-3958c59a5455 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:34.269590 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102f8de2-cdda-45c2-bd2a-3958c59a5455 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:34.288488 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102f8de2-cdda-45c2-bd2a-3958c59a5455 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:17:34.290128 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102f8de2-cdda-45c2-bd2a-3958c59a5455 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:17:34.308501 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b87fe2d-5b7b-42ba-93dc-e737801e87e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:34.309150 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b87fe2d-5b7b-42ba-93dc-e737801e87e9 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:34.309463 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b87fe2d-5b7b-42ba-93dc-e737801e87e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:34.309740 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b87fe2d-5b7b-42ba-93dc-e737801e87e9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:34.325136 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-102f8de2-cdda-45c2-bd2a-3958c59a5455 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 04:17:34.325136 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 04:17:34.325136 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 04:17:34.325136 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 04:17:34.325136 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 04:17:34.325136 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 04:17:34.325136 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 04:17:34.325136 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 04:17:34.325136 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 04:17:34.325136 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 04:17:34.325136 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 04:17:34.325136 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 04:17:34.325136 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 04:17:34.325136 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 04:17:34.325136 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 04:17:34.325136 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 04:17:34.326381 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 04:17:34.326381 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 04:17:34.326381 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 04:17:34.326381 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 04:17:34.326381 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 04:17:34.326381 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 04:17:34.326381 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 04:17:34.326381 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 04:17:34.326381 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 04:17:34.326381 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 04:17:34.326381 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 04:17:34.326381 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 04:17:34.326381 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 04:17:34.326381 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 04:17:34.326381 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 04:17:34.326381 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 04:17:34.326381 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 04:17:34.327261 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 04:17:34.327261 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 04:17:34.327261 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 04:17:34.328163 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 257/532] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Sep 6 04:17:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 374 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 04:17:34.333913 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102f8de2-cdda-45c2-bd2a-3958c59a5455 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:34.334206 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b87fe2d-5b7b-42ba-93dc-e737801e87e9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:17:34.335988 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b87fe2d-5b7b-42ba-93dc-e737801e87e9 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:17:34.352528 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-6b87fe2d-5b7b-42ba-93dc-e737801e87e9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 04:17:34.352528 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 04:17:34.352528 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 04:17:34.352528 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 04:17:34.352528 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 04:17:34.352528 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 04:17:34.352528 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 04:17:34.352528 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 04:17:34.352528 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 04:17:34.352528 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 04:17:34.352528 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 04:17:34.352528 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 04:17:34.352528 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 04:17:34.352528 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 04:17:34.352528 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 04:17:34.352528 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 04:17:34.353512 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 04:17:34.353512 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 04:17:34.353512 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 04:17:34.353512 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 04:17:34.353512 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 04:17:34.353512 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 04:17:34.353512 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 04:17:34.353512 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 04:17:34.353512 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 04:17:34.353512 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 04:17:34.353512 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 04:17:34.353512 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 04:17:34.353512 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 04:17:34.353512 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 04:17:34.353512 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 04:17:34.353512 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 04:17:34.353512 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 04:17:34.354384 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 04:17:34.354384 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 04:17:34.354384 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 04:17:34.356432 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 276/533] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Sep 6 04:17:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 04:17:34.365173 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b87fe2d-5b7b-42ba-93dc-e737801e87e9 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:34.484018 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b87fe2d-5b7b-42ba-93dc-e737801e87e9 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:34.515467 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:17:34.546198 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d7faa6-7620-4689-a06e-84b7443d7a37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:34.546821 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d7faa6-7620-4689-a06e-84b7443d7a37 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:34.548190 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102f8de2-cdda-45c2-bd2a-3958c59a5455 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:34.549032 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d7faa6-7620-4689-a06e-84b7443d7a37 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:34.549335 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d7faa6-7620-4689-a06e-84b7443d7a37 None admin] PATH_INFO: `/v3/domains` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:34.549994 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d7faa6-7620-4689-a06e-84b7443d7a37 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:17:34.550293 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d7faa6-7620-4689-a06e-84b7443d7a37 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:17:34.551779 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d7faa6-7620-4689-a06e-84b7443d7a37 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:17:34.575566 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 277/534] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Sep 6 04:17:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 04:17:34.576924 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0eab243-d8c9-4615-9043-b936c47aae1f None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:34.577212 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0eab243-d8c9-4615-9043-b936c47aae1f None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:34.577519 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0eab243-d8c9-4615-9043-b936c47aae1f None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:34.583122 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 278/535] 10.222.0.30 () {54 vars in 927 bytes} [Tue Sep 6 04:17:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 04:17:34.583933 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0eab243-d8c9-4615-9043-b936c47aae1f None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:34.604158 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f230946-32ee-4912-b32d-a75f1731454d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:34.604781 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f230946-32ee-4912-b32d-a75f1731454d None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:34.605061 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f230946-32ee-4912-b32d-a75f1731454d None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:34.605358 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f230946-32ee-4912-b32d-a75f1731454d None admin] PATH_INFO: `/v3/domains` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:34.606031 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f230946-32ee-4912-b32d-a75f1731454d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:17:34.606336 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f230946-32ee-4912-b32d-a75f1731454d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:17:34.607785 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f230946-32ee-4912-b32d-a75f1731454d None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:17:34.611067 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:17:34.625358 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 258/536] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Sep 6 04:17:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:17:34.626642 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03343d3b-4b11-4f66-a878-414016b7e42d None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:34.627254 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03343d3b-4b11-4f66-a878-414016b7e42d None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:34.627695 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03343d3b-4b11-4f66-a878-414016b7e42d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:34.646703 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-03343d3b-4b11-4f66-a878-414016b7e42d None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:17:34.648005 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-03343d3b-4b11-4f66-a878-414016b7e42d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:17:34.717951 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 259/537] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Sep 6 04:17:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 04:17:34.724782 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0eab243-d8c9-4615-9043-b936c47aae1f None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:34.726216 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03343d3b-4b11-4f66-a878-414016b7e42d None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:34.760510 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d9c503-31e5-4d7d-871f-fda6ad8ac7b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:34.761586 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d9c503-31e5-4d7d-871f-fda6ad8ac7b9 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:34.762038 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d9c503-31e5-4d7d-871f-fda6ad8ac7b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:34.762477 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d9c503-31e5-4d7d-871f-fda6ad8ac7b9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:34.770321 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d9c503-31e5-4d7d-871f-fda6ad8ac7b9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:17:34.772039 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d9c503-31e5-4d7d-871f-fda6ad8ac7b9 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:17:34.783706 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-73d9c503-31e5-4d7d-871f-fda6ad8ac7b9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 04:17:34.783706 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 04:17:34.783706 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 04:17:34.783706 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 04:17:34.783706 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 04:17:34.783706 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 04:17:34.783706 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 04:17:34.783706 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 04:17:34.783706 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 04:17:34.783706 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 04:17:34.783706 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 04:17:34.783706 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 04:17:34.783706 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 06 04:17:34.783706 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 06 04:17:34.783706 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 06 04:17:34.784554 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 06 04:17:34.784554 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 04:17:34.784554 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 04:17:34.784554 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 04:17:34.784554 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 04:17:34.784554 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 04:17:34.784554 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 04:17:34.784554 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 04:17:34.784554 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 04:17:34.784554 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 04:17:34.784554 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 04:17:34.784554 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 04:17:34.784554 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 04:17:34.784554 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 04:17:34.784554 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 04:17:34.784554 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 04:17:34.784554 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 04:17:34.785613 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 06 04:17:34.785613 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 06 04:17:34.785613 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 06 04:17:34.785613 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 06 04:17:34.785613 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 06 04:17:34.785613 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 06 04:17:34.785613 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 04:17:34.785613 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 04:17:34.787650 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 279/538] 10.222.0.30 () {56 vars in 1111 bytes} [Tue Sep 6 04:17:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 04:17:34.793196 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d9c503-31e5-4d7d-871f-fda6ad8ac7b9 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:34.855556 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:17:34.909726 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03343d3b-4b11-4f66-a878-414016b7e42d None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:34.942708 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:17:34.982710 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4765854-86f4-46ab-a1af-5a76f4fbc191 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:34.983819 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4765854-86f4-46ab-a1af-5a76f4fbc191 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:34.984342 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4765854-86f4-46ab-a1af-5a76f4fbc191 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:34.984807 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4765854-86f4-46ab-a1af-5a76f4fbc191 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:34.992807 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d9c503-31e5-4d7d-871f-fda6ad8ac7b9 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:35.003124 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4765854-86f4-46ab-a1af-5a76f4fbc191 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:17:35.006063 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4765854-86f4-46ab-a1af-5a76f4fbc191 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:17:35.026398 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-d4765854-86f4-46ab-a1af-5a76f4fbc191 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 04:17:35.026398 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 04:17:35.026398 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 04:17:35.026398 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 04:17:35.026398 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 04:17:35.026398 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 04:17:35.026398 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 04:17:35.026398 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 04:17:35.026398 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 04:17:35.026398 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 04:17:35.026398 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 04:17:35.026398 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 04:17:35.026398 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 06 04:17:35.026398 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 06 04:17:35.026398 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 06 04:17:35.027781 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 06 04:17:35.027781 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 04:17:35.027781 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 04:17:35.027781 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 04:17:35.027781 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 04:17:35.027781 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 04:17:35.027781 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 04:17:35.027781 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 04:17:35.027781 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 04:17:35.027781 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 04:17:35.027781 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 04:17:35.027781 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 04:17:35.027781 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 04:17:35.027781 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 04:17:35.027781 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 04:17:35.027781 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 04:17:35.027781 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 04:17:35.029181 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 06 04:17:35.029181 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 06 04:17:35.029181 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 06 04:17:35.029181 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 06 04:17:35.029181 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 06 04:17:35.029181 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 06 04:17:35.029181 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 04:17:35.029181 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 04:17:35.033692 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a34dcef-f19c-46c2-a393-fe91e0cee56e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:35.034267 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a34dcef-f19c-46c2-a393-fe91e0cee56e None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:35.034511 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a34dcef-f19c-46c2-a393-fe91e0cee56e None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:35.034748 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 260/539] 10.222.0.30 () {56 vars in 1111 bytes} [Tue Sep 6 04:17:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 316 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 04:17:35.034893 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a34dcef-f19c-46c2-a393-fe91e0cee56e None admin] PATH_INFO: `/v3/projects` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:35.040074 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a34dcef-f19c-46c2-a393-fe91e0cee56e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:17:35.040074 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a34dcef-f19c-46c2-a393-fe91e0cee56e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:17:35.042098 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a34dcef-f19c-46c2-a393-fe91e0cee56e None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:17:35.043493 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4765854-86f4-46ab-a1af-5a76f4fbc191 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:35.057347 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 280/540] 10.222.0.30 () {56 vars in 1148 bytes} [Tue Sep 6 04:17:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:17:35.061251 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a34dcef-f19c-46c2-a393-fe91e0cee56e None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:35.204278 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4765854-86f4-46ab-a1af-5a76f4fbc191 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:35.211290 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a34dcef-f19c-46c2-a393-fe91e0cee56e None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:35.245979 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:17:35.248747 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637a0391-4b03-45d8-9d07-9bceff8d6689 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:35.249504 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637a0391-4b03-45d8-9d07-9bceff8d6689 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:35.249891 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637a0391-4b03-45d8-9d07-9bceff8d6689 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:35.250310 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637a0391-4b03-45d8-9d07-9bceff8d6689 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:35.251134 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637a0391-4b03-45d8-9d07-9bceff8d6689 None admin] RBAC: Adding query filter params (role.id=83cf0a194f8d45e792411e2d9afdfe50, scope.project.id=b707733ce65c42edbc1fd95ad0a1c06c, user.id=d09b5ab675634a2eb4b03755743f7d0d) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:17:35.251561 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637a0391-4b03-45d8-9d07-9bceff8d6689 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:17:35.252543 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7468d4ff-5b02-4c2b-96b7-33d00c019902 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:35.253328 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7468d4ff-5b02-4c2b-96b7-33d00c019902 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:35.253606 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637a0391-4b03-45d8-9d07-9bceff8d6689 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:17:35.254074 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7468d4ff-5b02-4c2b-96b7-33d00c019902 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:35.254512 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7468d4ff-5b02-4c2b-96b7-33d00c019902 None admin] PATH_INFO: `/v3/projects` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:35.255356 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7468d4ff-5b02-4c2b-96b7-33d00c019902 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:17:35.255844 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7468d4ff-5b02-4c2b-96b7-33d00c019902 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:17:35.258025 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7468d4ff-5b02-4c2b-96b7-33d00c019902 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:17:35.269169 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 281/541] 10.222.0.30 () {56 vars in 1374 bytes} [Tue Sep 6 04:17:35 2022] GET /identity/v3/role_assignments?user.id=d09b5ab675634a2eb4b03755743f7d0d&scope.project.id=b707733ce65c42edbc1fd95ad0a1c06c&role.id=83cf0a194f8d45e792411e2d9afdfe50 => generated 259 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 04:17:35.269169 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637a0391-4b03-45d8-9d07-9bceff8d6689 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:35.273446 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:17:35.287420 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 261/542] 10.222.0.30 () {56 vars in 1148 bytes} [Tue Sep 6 04:17:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 04:17:35.292369 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7468d4ff-5b02-4c2b-96b7-33d00c019902 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:35.481982 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637a0391-4b03-45d8-9d07-9bceff8d6689 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:35.490104 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7468d4ff-5b02-4c2b-96b7-33d00c019902 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:35.524974 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93edcd5b-b132-4238-b253-752d9b3333aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:35.526683 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93edcd5b-b132-4238-b253-752d9b3333aa None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:35.526967 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93edcd5b-b132-4238-b253-752d9b3333aa None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:35.527238 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93edcd5b-b132-4238-b253-752d9b3333aa None admin] PATH_INFO: `/v3/roles/service` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:35.537077 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30eff79-ba88-4cb9-ba3b-b5c756f2f961 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:35.538135 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30eff79-ba88-4cb9-ba3b-b5c756f2f961 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:35.538608 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30eff79-ba88-4cb9-ba3b-b5c756f2f961 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:35.539069 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30eff79-ba88-4cb9-ba3b-b5c756f2f961 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:35.540851 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30eff79-ba88-4cb9-ba3b-b5c756f2f961 None admin] RBAC: Adding query filter params (role.id=83cf0a194f8d45e792411e2d9afdfe50, scope.project.id=b707733ce65c42edbc1fd95ad0a1c06c, user.id=90492440f6a6471e8392361c098428aa) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:17:35.541637 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30eff79-ba88-4cb9-ba3b-b5c756f2f961 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:17:35.544334 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30eff79-ba88-4cb9-ba3b-b5c756f2f961 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:17:35.552214 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 262/543] 10.222.0.30 () {56 vars in 1374 bytes} [Tue Sep 6 04:17:35 2022] GET /identity/v3/role_assignments?user.id=90492440f6a6471e8392361c098428aa&scope.project.id=b707733ce65c42edbc1fd95ad0a1c06c&role.id=83cf0a194f8d45e792411e2d9afdfe50 => generated 259 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 04:17:35.562701 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93edcd5b-b132-4238-b253-752d9b3333aa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 04:17:35.569097 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-93edcd5b-b132-4238-b253-752d9b3333aa None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 06 04:17:35.569097 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 04:17:35.569097 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 04:17:35.569097 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 04:17:35.569097 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 04:17:35.569097 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 04:17:35.569097 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 04:17:35.569097 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 04:17:35.569097 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 04:17:35.569097 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 04:17:35.569097 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 04:17:35.569097 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 04:17:35.569097 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 04:17:35.569097 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 04:17:35.569097 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 04:17:35.569097 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 04:17:35.570036 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 04:17:35.570036 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 04:17:35.570036 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 04:17:35.570036 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 04:17:35.570036 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 04:17:35.570036 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 04:17:35.570036 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 04:17:35.570036 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 04:17:35.570036 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 04:17:35.570036 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 04:17:35.570036 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 04:17:35.570036 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 04:17:35.570036 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 04:17:35.570036 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 04:17:35.570036 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 04:17:35.570036 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 04:17:35.571411 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 04:17:35.571411 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 04:17:35.571411 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 04:17:35.571411 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 04:17:35.571411 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 04:17:35.571411 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 04:17:35.571411 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 04:17:35.571411 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 04:17:35.571411 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 04:17:35.571411 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 04:17:35.571411 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 06 04:17:35.571411 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 04:17:35.571411 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 282/544] 10.222.0.30 () {56 vars in 1102 bytes} [Tue Sep 6 04:17:35 2022] GET /identity/v3/roles/service => generated 85 bytes in 306 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 04:17:35.576294 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30eff79-ba88-4cb9-ba3b-b5c756f2f961 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:35.663606 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:17:35.714422 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30eff79-ba88-4cb9-ba3b-b5c756f2f961 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:35.763628 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7abb54-daa1-4c8b-9997-fc33476f04f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:35.764633 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7abb54-daa1-4c8b-9997-fc33476f04f2 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:35.765172 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7abb54-daa1-4c8b-9997-fc33476f04f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:35.765652 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7abb54-daa1-4c8b-9997-fc33476f04f2 None admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:35.766569 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d7abb54-daa1-4c8b-9997-fc33476f04f2 None admin] RBAC: Adding query filter params (name=service) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:17:35.767084 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d7abb54-daa1-4c8b-9997-fc33476f04f2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:17:35.768649 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d7abb54-daa1-4c8b-9997-fc33476f04f2 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:17:35.778994 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 263/545] 10.222.0.30 () {56 vars in 1103 bytes} [Tue Sep 6 04:17:35 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:17:35.784545 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93edcd5b-b132-4238-b253-752d9b3333aa None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:35.854922 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:17:35.906347 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93edcd5b-b132-4238-b253-752d9b3333aa None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:35.954118 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d57527-54df-4406-906b-ab842808e577 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:35.954754 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d57527-54df-4406-906b-ab842808e577 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:35.955030 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d57527-54df-4406-906b-ab842808e577 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:35.955299 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d57527-54df-4406-906b-ab842808e577 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:35.970927 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54563df-14b7-42ac-9900-174637440693 None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:35.971314 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54563df-14b7-42ac-9900-174637440693 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:35.971661 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54563df-14b7-42ac-9900-174637440693 None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:35.974260 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 264/546] 10.222.0.30 () {54 vars in 927 bytes} [Tue Sep 6 04:17:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 06 04:17:35.978808 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d57527-54df-4406-906b-ab842808e577 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:17:35.979120 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234667b1-cef7-44a6-8c54-bb15c9d3e887 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:35.979408 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234667b1-cef7-44a6-8c54-bb15c9d3e887 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:35.979679 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234667b1-cef7-44a6-8c54-bb15c9d3e887 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:35.981061 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d57527-54df-4406-906b-ab842808e577 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:17:36.010321 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-e1d57527-54df-4406-906b-ab842808e577 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 04:17:36.010321 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 04:17:36.010321 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 04:17:36.010321 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 04:17:36.010321 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 04:17:36.010321 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 04:17:36.010321 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 04:17:36.010321 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 04:17:36.010321 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 04:17:36.010321 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 04:17:36.010321 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 04:17:36.010321 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 04:17:36.010321 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 04:17:36.010321 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 04:17:36.010321 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 04:17:36.010321 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 04:17:36.011359 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 04:17:36.011359 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 04:17:36.011359 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 04:17:36.011359 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 04:17:36.011359 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 04:17:36.011359 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 04:17:36.011359 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 04:17:36.011359 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 04:17:36.011359 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 04:17:36.011359 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 04:17:36.011359 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 04:17:36.011359 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 04:17:36.011359 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 04:17:36.011359 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 04:17:36.011359 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 04:17:36.011359 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 04:17:36.011359 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 04:17:36.012189 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 04:17:36.012189 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 04:17:36.012189 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 04:17:36.012528 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-234667b1-cef7-44a6-8c54-bb15c9d3e887 None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:17:36.013953 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-234667b1-cef7-44a6-8c54-bb15c9d3e887 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:17:36.015722 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 283/547] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Sep 6 04:17:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 04:17:36.018462 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d57527-54df-4406-906b-ab842808e577 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:36.156304 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 265/548] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Sep 6 04:17:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 06 04:17:36.180344 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d57527-54df-4406-906b-ab842808e577 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:36.202230 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:17:36.203645 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416f7c22-0a3b-4eef-a9f3-989e1f6473da None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:36.203937 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416f7c22-0a3b-4eef-a9f3-989e1f6473da None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:36.204218 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416f7c22-0a3b-4eef-a9f3-989e1f6473da None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:36.205846 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 266/549] 10.222.0.30 () {54 vars in 927 bytes} [Tue Sep 6 04:17:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 06 04:17:36.211069 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2996535a-05ad-4465-800b-f7173d7df93a None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:36.211357 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2996535a-05ad-4465-800b-f7173d7df93a None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:36.211633 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2996535a-05ad-4465-800b-f7173d7df93a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:36.225103 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3a513a-cb42-41fc-998a-39dcd856c689 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:36.225773 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3a513a-cb42-41fc-998a-39dcd856c689 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:36.226058 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3a513a-cb42-41fc-998a-39dcd856c689 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:36.226327 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3a513a-cb42-41fc-998a-39dcd856c689 None admin] PATH_INFO: `/v3/domains` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:36.226966 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c3a513a-cb42-41fc-998a-39dcd856c689 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:17:36.227262 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c3a513a-cb42-41fc-998a-39dcd856c689 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:17:36.228621 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c3a513a-cb42-41fc-998a-39dcd856c689 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:17:36.233740 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-2996535a-05ad-4465-800b-f7173d7df93a None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:17:36.235178 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-2996535a-05ad-4465-800b-f7173d7df93a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:17:36.241377 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 284/550] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Sep 6 04:17:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:17:36.250357 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3a513a-cb42-41fc-998a-39dcd856c689 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:36.320584 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:17:36.368488 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 267/551] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Sep 6 04:17:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 06 04:17:36.412116 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3a513a-cb42-41fc-998a-39dcd856c689 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:36.447507 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c139ea1e-9c71-4937-99e0-27abeb3e4eea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:36.448140 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c139ea1e-9c71-4937-99e0-27abeb3e4eea None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:36.448417 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c139ea1e-9c71-4937-99e0-27abeb3e4eea None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:36.448686 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c139ea1e-9c71-4937-99e0-27abeb3e4eea None admin] PATH_INFO: `/v3/users/swift` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:36.456244 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c139ea1e-9c71-4937-99e0-27abeb3e4eea None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:17:36.467257 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c139ea1e-9c71-4937-99e0-27abeb3e4eea None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:17:36.471472 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-c139ea1e-9c71-4937-99e0-27abeb3e4eea None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 06 04:17:36.471472 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 04:17:36.471472 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 04:17:36.471472 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 04:17:36.471472 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 04:17:36.471472 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 04:17:36.471472 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 04:17:36.471472 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 04:17:36.471472 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 04:17:36.471472 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 04:17:36.471472 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 04:17:36.471472 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 04:17:36.471472 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 06 04:17:36.471472 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 06 04:17:36.471472 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 06 04:17:36.471472 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 06 04:17:36.472376 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 04:17:36.472376 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 04:17:36.472376 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 06 04:17:36.472376 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 04:17:36.472376 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 06 04:17:36.472376 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 04:17:36.472376 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 04:17:36.472376 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 04:17:36.472376 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 04:17:36.472376 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 04:17:36.472376 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 04:17:36.472376 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 04:17:36.472376 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 04:17:36.472376 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 04:17:36.472376 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 04:17:36.472376 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 04:17:36.472376 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 04:17:36.472376 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 04:17:36.473289 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 04:17:36.473289 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 04:17:36.473289 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 06 04:17:36.473289 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 06 04:17:36.473289 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 06 04:17:36.473289 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 06 04:17:36.473289 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 06 04:17:36.473289 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 06 04:17:36.473289 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 06 04:17:36.473289 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 04:17:36.473289 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 285/552] 10.222.0.30 () {56 vars in 1096 bytes} [Tue Sep 6 04:17:36 2022] GET /identity/v3/users/swift => generated 83 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 04:17:36.480061 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2996535a-05ad-4465-800b-f7173d7df93a None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:36.490757 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c139ea1e-9c71-4937-99e0-27abeb3e4eea None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:36.549730 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:17:36.600447 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c139ea1e-9c71-4937-99e0-27abeb3e4eea None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:36.640751 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb14c5b-6c88-4c48-b344-250deef00001 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:36.641515 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb14c5b-6c88-4c48-b344-250deef00001 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:36.641871 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb14c5b-6c88-4c48-b344-250deef00001 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:36.642215 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb14c5b-6c88-4c48-b344-250deef00001 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:36.654026 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb14c5b-6c88-4c48-b344-250deef00001 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 04:17:36.660920 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-8fb14c5b-6c88-4c48-b344-250deef00001 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 06 04:17:36.660920 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 04:17:36.660920 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 04:17:36.660920 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 04:17:36.660920 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 04:17:36.660920 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 04:17:36.660920 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 04:17:36.660920 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 04:17:36.660920 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 04:17:36.660920 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 04:17:36.660920 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 04:17:36.660920 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 04:17:36.660920 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 04:17:36.660920 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 04:17:36.660920 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 04:17:36.660920 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 04:17:36.662357 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 04:17:36.662357 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 04:17:36.662357 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 04:17:36.662357 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 04:17:36.662357 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 04:17:36.662357 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 04:17:36.662357 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 04:17:36.662357 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 04:17:36.662357 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 04:17:36.662357 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 04:17:36.662357 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 04:17:36.662357 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 04:17:36.662357 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 04:17:36.662357 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 04:17:36.662357 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 04:17:36.662357 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 04:17:36.663433 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 04:17:36.663433 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 04:17:36.663433 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 04:17:36.663433 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 04:17:36.663433 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 04:17:36.663433 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 04:17:36.663433 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 04:17:36.663433 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 04:17:36.663433 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 04:17:36.663433 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 04:17:36.663433 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 06 04:17:36.663433 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 04:17:36.667139 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 286/553] 10.222.0.30 () {56 vars in 1102 bytes} [Tue Sep 6 04:17:36 2022] GET /identity/v3/roles/service => generated 85 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 04:17:36.669153 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb14c5b-6c88-4c48-b344-250deef00001 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:36.698808 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2996535a-05ad-4465-800b-f7173d7df93a None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:36.774948 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cee9e64-f16d-42ec-92ff-2f02709b6071 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:36.775728 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cee9e64-f16d-42ec-92ff-2f02709b6071 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:36.776101 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cee9e64-f16d-42ec-92ff-2f02709b6071 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:36.776489 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cee9e64-f16d-42ec-92ff-2f02709b6071 None admin] PATH_INFO: `/v3/users` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:36.777334 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cee9e64-f16d-42ec-92ff-2f02709b6071 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:17:36.777764 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cee9e64-f16d-42ec-92ff-2f02709b6071 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:17:36.779297 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cee9e64-f16d-42ec-92ff-2f02709b6071 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:17:36.786913 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb14c5b-6c88-4c48-b344-250deef00001 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:36.789775 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:17:36.801457 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 268/554] 10.222.0.30 () {56 vars in 1135 bytes} [Tue Sep 6 04:17:36 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 322 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:17:36.802604 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cee9e64-f16d-42ec-92ff-2f02709b6071 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:36.845909 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70986cca-fec5-483b-916e-9af0417fcbed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:36.848071 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70986cca-fec5-483b-916e-9af0417fcbed None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:36.848528 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70986cca-fec5-483b-916e-9af0417fcbed None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:36.849203 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70986cca-fec5-483b-916e-9af0417fcbed None admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:36.850074 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70986cca-fec5-483b-916e-9af0417fcbed None admin] RBAC: Adding query filter params (name=service) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:17:36.850602 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70986cca-fec5-483b-916e-9af0417fcbed None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:17:36.852714 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:17:36.854396 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70986cca-fec5-483b-916e-9af0417fcbed None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:17:36.864788 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 287/555] 10.222.0.30 () {56 vars in 1103 bytes} [Tue Sep 6 04:17:36 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 04:17:36.871428 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70986cca-fec5-483b-916e-9af0417fcbed None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:36.937893 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cee9e64-f16d-42ec-92ff-2f02709b6071 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:36.977568 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575c7b4e-c105-4bd6-8abe-e353289a5d9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:36.978214 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575c7b4e-c105-4bd6-8abe-e353289a5d9b None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:36.978492 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575c7b4e-c105-4bd6-8abe-e353289a5d9b None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:36.978764 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575c7b4e-c105-4bd6-8abe-e353289a5d9b None admin] PATH_INFO: `/v3/roles/service` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:36.989457 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-575c7b4e-c105-4bd6-8abe-e353289a5d9b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 04:17:36.994525 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-575c7b4e-c105-4bd6-8abe-e353289a5d9b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 06 04:17:36.994525 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 04:17:36.994525 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 04:17:36.994525 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 04:17:36.994525 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 04:17:36.994525 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 04:17:36.994525 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 04:17:36.994525 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 04:17:36.994525 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 04:17:36.994525 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 04:17:36.994525 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 04:17:36.994525 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 04:17:36.994525 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 04:17:36.994525 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 04:17:36.994525 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 04:17:36.994525 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 04:17:36.995421 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 04:17:36.995421 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 04:17:36.995421 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 04:17:36.995421 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 04:17:36.995421 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 04:17:36.995421 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 04:17:36.995421 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 04:17:36.995421 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 04:17:36.995421 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 04:17:36.995421 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 04:17:36.995421 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 04:17:36.995421 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 04:17:36.995421 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 04:17:36.995421 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 04:17:36.995421 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 04:17:36.995421 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 04:17:36.996940 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 04:17:36.996940 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 04:17:36.996940 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 04:17:36.996940 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 04:17:36.996940 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 04:17:36.996940 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 04:17:36.996940 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 04:17:36.996940 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 04:17:36.996940 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 04:17:36.996940 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 04:17:36.996940 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 06 04:17:36.996940 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 04:17:36.996940 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 269/556] 10.222.0.30 () {56 vars in 1102 bytes} [Tue Sep 6 04:17:36 2022] GET /identity/v3/roles/service => generated 85 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 04:17:36.996940 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575c7b4e-c105-4bd6-8abe-e353289a5d9b None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:37.016950 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70986cca-fec5-483b-916e-9af0417fcbed None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:37.054738 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac037bfa-b1ed-49cd-b6ec-bd07984b10e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:37.055779 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac037bfa-b1ed-49cd-b6ec-bd07984b10e3 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:37.056263 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac037bfa-b1ed-49cd-b6ec-bd07984b10e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:37.056723 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac037bfa-b1ed-49cd-b6ec-bd07984b10e3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:37.085640 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:17:37.097987 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:17:37.103498 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575c7b4e-c105-4bd6-8abe-e353289a5d9b None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:37.135210 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac037bfa-b1ed-49cd-b6ec-bd07984b10e3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:17:37.137383 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac037bfa-b1ed-49cd-b6ec-bd07984b10e3 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:17:37.139051 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c89887-3cea-41a8-9eb9-1ba3201724b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:37.139953 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c89887-3cea-41a8-9eb9-1ba3201724b1 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:37.140337 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c89887-3cea-41a8-9eb9-1ba3201724b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:37.140784 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c89887-3cea-41a8-9eb9-1ba3201724b1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:37.152622 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c89887-3cea-41a8-9eb9-1ba3201724b1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:17:37.154228 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c89887-3cea-41a8-9eb9-1ba3201724b1 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:17:37.169807 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-60c89887-3cea-41a8-9eb9-1ba3201724b1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 04:17:37.169807 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 04:17:37.169807 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 04:17:37.169807 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 04:17:37.169807 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 04:17:37.169807 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 04:17:37.169807 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 04:17:37.169807 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 04:17:37.169807 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 04:17:37.169807 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 04:17:37.169807 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 04:17:37.169807 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 04:17:37.169807 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 04:17:37.169807 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 04:17:37.169807 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 04:17:37.169807 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 04:17:37.170791 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 04:17:37.170791 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 04:17:37.170791 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 04:17:37.170791 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 04:17:37.170791 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 04:17:37.170791 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 04:17:37.170791 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 04:17:37.170791 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 04:17:37.170791 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 04:17:37.170791 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 04:17:37.170791 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 04:17:37.170791 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 04:17:37.170791 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 04:17:37.170791 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 04:17:37.170791 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 04:17:37.170791 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 04:17:37.170791 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 04:17:37.171688 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 04:17:37.171688 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 04:17:37.171688 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 04:17:37.175947 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 270/557] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Sep 6 04:17:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 04:17:37.176863 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c89887-3cea-41a8-9eb9-1ba3201724b1 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:37.255199 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 288/558] 10.222.0.30 () {58 vars in 1282 bytes} [Tue Sep 6 04:17:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 06 04:17:37.256495 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac037bfa-b1ed-49cd-b6ec-bd07984b10e3 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:37.280723 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c89887-3cea-41a8-9eb9-1ba3201724b1 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:37.318159 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb71d30-394f-49e2-991a-02a7140fc9f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:37.318856 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb71d30-394f-49e2-991a-02a7140fc9f3 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:37.319205 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb71d30-394f-49e2-991a-02a7140fc9f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:37.319551 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb71d30-394f-49e2-991a-02a7140fc9f3 None admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:37.320280 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb71d30-394f-49e2-991a-02a7140fc9f3 None admin] RBAC: Adding query filter params (name=service) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:17:37.320650 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb71d30-394f-49e2-991a-02a7140fc9f3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:17:37.322123 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb71d30-394f-49e2-991a-02a7140fc9f3 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:17:37.323019 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:17:37.330884 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 271/559] 10.222.0.30 () {56 vars in 1103 bytes} [Tue Sep 6 04:17:37 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:17:37.331829 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb71d30-394f-49e2-991a-02a7140fc9f3 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:37.344760 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:17:37.373702 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac037bfa-b1ed-49cd-b6ec-bd07984b10e3 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:37.413603 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610eb1b8-ec38-4ef8-b5a4-ebd80722bca7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:37.414846 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610eb1b8-ec38-4ef8-b5a4-ebd80722bca7 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:37.415313 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610eb1b8-ec38-4ef8-b5a4-ebd80722bca7 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:37.415741 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610eb1b8-ec38-4ef8-b5a4-ebd80722bca7 None admin] PATH_INFO: `/v3/domains` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:37.416540 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-610eb1b8-ec38-4ef8-b5a4-ebd80722bca7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:17:37.417019 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-610eb1b8-ec38-4ef8-b5a4-ebd80722bca7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:17:37.418707 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-610eb1b8-ec38-4ef8-b5a4-ebd80722bca7 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:17:37.433597 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 289/560] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Sep 6 04:17:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:17:37.434955 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610eb1b8-ec38-4ef8-b5a4-ebd80722bca7 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:37.451251 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb71d30-394f-49e2-991a-02a7140fc9f3 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:37.487389 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4f59d4-8a55-4696-9e86-9f2c0dbb3b26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:37.488147 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4f59d4-8a55-4696-9e86-9f2c0dbb3b26 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:37.488489 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4f59d4-8a55-4696-9e86-9f2c0dbb3b26 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:37.488765 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4f59d4-8a55-4696-9e86-9f2c0dbb3b26 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:37.500805 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4f59d4-8a55-4696-9e86-9f2c0dbb3b26 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:17:37.502560 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4f59d4-8a55-4696-9e86-9f2c0dbb3b26 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:17:37.517853 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-9e4f59d4-8a55-4696-9e86-9f2c0dbb3b26 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 04:17:37.517853 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 04:17:37.517853 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 04:17:37.517853 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 04:17:37.517853 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 04:17:37.517853 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 04:17:37.517853 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 04:17:37.517853 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 04:17:37.517853 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 04:17:37.517853 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 04:17:37.517853 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 04:17:37.517853 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 04:17:37.517853 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 04:17:37.517853 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 04:17:37.517853 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 04:17:37.517853 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 04:17:37.520193 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 04:17:37.520193 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 04:17:37.520193 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 04:17:37.520193 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 04:17:37.520193 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 04:17:37.520193 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 04:17:37.520193 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 04:17:37.520193 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 04:17:37.520193 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 04:17:37.520193 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 04:17:37.520193 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 04:17:37.520193 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 04:17:37.520193 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 04:17:37.520193 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 04:17:37.520193 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 04:17:37.520193 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 04:17:37.520193 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 04:17:37.521095 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 04:17:37.521095 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 04:17:37.521095 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 04:17:37.521095 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 272/561] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Sep 6 04:17:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 04:17:37.521095 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4f59d4-8a55-4696-9e86-9f2c0dbb3b26 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:37.557731 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610eb1b8-ec38-4ef8-b5a4-ebd80722bca7 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:37.575588 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:17:37.594435 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b62ddb9-d176-4821-b1d4-554c70603c8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:37.595069 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b62ddb9-d176-4821-b1d4-554c70603c8a None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:37.595349 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b62ddb9-d176-4821-b1d4-554c70603c8a None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:37.595622 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b62ddb9-d176-4821-b1d4-554c70603c8a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:37.596223 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:17:37.607483 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b62ddb9-d176-4821-b1d4-554c70603c8a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:17:37.608931 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b62ddb9-d176-4821-b1d4-554c70603c8a None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:17:37.624091 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-2b62ddb9-d176-4821-b1d4-554c70603c8a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 04:17:37.624091 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 04:17:37.624091 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 04:17:37.624091 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 04:17:37.624091 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 04:17:37.624091 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 04:17:37.624091 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 04:17:37.624091 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 04:17:37.624091 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 04:17:37.624091 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 04:17:37.624091 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 04:17:37.624091 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 04:17:37.624091 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 04:17:37.624091 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 04:17:37.624091 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 04:17:37.624091 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 04:17:37.625106 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 04:17:37.625106 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 04:17:37.625106 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 04:17:37.625106 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 04:17:37.625106 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 04:17:37.625106 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 04:17:37.625106 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 04:17:37.625106 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 04:17:37.625106 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 04:17:37.625106 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 04:17:37.625106 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 04:17:37.625106 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 04:17:37.625106 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 04:17:37.625106 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 04:17:37.625106 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 04:17:37.625106 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 04:17:37.625106 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 04:17:37.626052 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 04:17:37.626052 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 04:17:37.626052 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 04:17:37.626052 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 290/562] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Sep 6 04:17:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 04:17:37.626052 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b62ddb9-d176-4821-b1d4-554c70603c8a None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:37.641081 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4f59d4-8a55-4696-9e86-9f2c0dbb3b26 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:37.680212 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f5b41f-cd40-4c8c-8772-db58fea25c8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:37.680853 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f5b41f-cd40-4c8c-8772-db58fea25c8b None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:37.681126 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f5b41f-cd40-4c8c-8772-db58fea25c8b None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:37.681425 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f5b41f-cd40-4c8c-8772-db58fea25c8b None admin] PATH_INFO: `/v3/users/nova` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:37.689126 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f5b41f-cd40-4c8c-8772-db58fea25c8b None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:17:37.690594 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f5b41f-cd40-4c8c-8772-db58fea25c8b None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:17:37.704141 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-00f5b41f-cd40-4c8c-8772-db58fea25c8b None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 06 04:17:37.704141 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 04:17:37.704141 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 04:17:37.704141 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 04:17:37.704141 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 04:17:37.704141 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 04:17:37.704141 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 04:17:37.704141 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 04:17:37.704141 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 04:17:37.704141 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 04:17:37.704141 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 04:17:37.704141 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 04:17:37.704141 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 06 04:17:37.704141 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 06 04:17:37.704141 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 06 04:17:37.704141 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 06 04:17:37.705155 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 04:17:37.705155 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 04:17:37.705155 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 06 04:17:37.705155 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 04:17:37.705155 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 06 04:17:37.705155 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 04:17:37.705155 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 04:17:37.705155 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 04:17:37.705155 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 04:17:37.705155 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 04:17:37.705155 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 04:17:37.705155 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 04:17:37.705155 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 04:17:37.705155 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 04:17:37.705155 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 04:17:37.705155 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 04:17:37.705155 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 04:17:37.705155 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 04:17:37.706163 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 04:17:37.706163 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 04:17:37.706163 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 06 04:17:37.706163 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 06 04:17:37.706163 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 06 04:17:37.706163 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 06 04:17:37.706163 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 06 04:17:37.706163 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 06 04:17:37.706163 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 06 04:17:37.706163 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 04:17:37.706163 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 273/563] 10.222.0.30 () {56 vars in 1093 bytes} [Tue Sep 6 04:17:37 2022] GET /identity/v3/users/nova => generated 82 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 04:17:37.706857 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86864411-ecd0-48a7-a1c1-57858124d04e None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:37.706857 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86864411-ecd0-48a7-a1c1-57858124d04e None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:37.707891 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86864411-ecd0-48a7-a1c1-57858124d04e None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:37.713385 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 274/564] 10.222.0.30 () {54 vars in 927 bytes} [Tue Sep 6 04:17:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 04:17:37.713385 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86864411-ecd0-48a7-a1c1-57858124d04e None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:37.756041 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b62ddb9-d176-4821-b1d4-554c70603c8a None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:37.791458 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b159561c-fd77-4903-8686-9b4c4e0b0b35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:37.792098 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b159561c-fd77-4903-8686-9b4c4e0b0b35 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:37.792385 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b159561c-fd77-4903-8686-9b4c4e0b0b35 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:37.792656 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b159561c-fd77-4903-8686-9b4c4e0b0b35 None admin] PATH_INFO: `/v3/domains` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:37.793399 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b159561c-fd77-4903-8686-9b4c4e0b0b35 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:17:37.793708 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b159561c-fd77-4903-8686-9b4c4e0b0b35 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:17:37.795147 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b159561c-fd77-4903-8686-9b4c4e0b0b35 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:17:37.807665 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 291/565] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Sep 6 04:17:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:17:37.808304 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b159561c-fd77-4903-8686-9b4c4e0b0b35 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:37.831207 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86864411-ecd0-48a7-a1c1-57858124d04e None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:37.840279 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:17:37.854459 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:17:37.870866 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291e8c17-e68f-4f1b-b32c-5e338b1dfa96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:37.871534 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291e8c17-e68f-4f1b-b32c-5e338b1dfa96 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:37.871816 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291e8c17-e68f-4f1b-b32c-5e338b1dfa96 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:37.872083 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291e8c17-e68f-4f1b-b32c-5e338b1dfa96 None admin] PATH_INFO: `/v3/domains` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:37.872772 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291e8c17-e68f-4f1b-b32c-5e338b1dfa96 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:17:37.873256 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291e8c17-e68f-4f1b-b32c-5e338b1dfa96 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:17:37.874671 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291e8c17-e68f-4f1b-b32c-5e338b1dfa96 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:17:37.891245 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 275/566] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Sep 6 04:17:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:17:37.893220 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac17829a-b5c8-44f5-98a2-b51393a615dc None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:37.893525 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac17829a-b5c8-44f5-98a2-b51393a615dc None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:37.893793 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac17829a-b5c8-44f5-98a2-b51393a615dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:37.911950 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-ac17829a-b5c8-44f5-98a2-b51393a615dc None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:17:37.913216 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-ac17829a-b5c8-44f5-98a2-b51393a615dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:17:37.920364 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b159561c-fd77-4903-8686-9b4c4e0b0b35 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:37.954721 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e37d7e9-a64a-48c7-9887-bf069dc53b95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:37.955338 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e37d7e9-a64a-48c7-9887-bf069dc53b95 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:37.955621 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e37d7e9-a64a-48c7-9887-bf069dc53b95 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:37.955894 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e37d7e9-a64a-48c7-9887-bf069dc53b95 None admin] PATH_INFO: `/v3/users` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:37.956622 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e37d7e9-a64a-48c7-9887-bf069dc53b95 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:17:37.957532 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e37d7e9-a64a-48c7-9887-bf069dc53b95 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:17:37.958976 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e37d7e9-a64a-48c7-9887-bf069dc53b95 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:17:37.980828 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 292/567] 10.222.0.30 () {56 vars in 1133 bytes} [Tue Sep 6 04:17:37 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:17:37.980828 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e37d7e9-a64a-48c7-9887-bf069dc53b95 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:37.985705 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 276/568] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Sep 6 04:17:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 04:17:37.987036 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a254f6-66e9-4eb0-a7b5-d65492cd3159 None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:37.987327 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a254f6-66e9-4eb0-a7b5-d65492cd3159 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:37.987598 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a254f6-66e9-4eb0-a7b5-d65492cd3159 None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:37.989437 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 277/569] 10.222.0.30 () {54 vars in 927 bytes} [Tue Sep 6 04:17:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 04:17:37.990886 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a254f6-66e9-4eb0-a7b5-d65492cd3159 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:38.090657 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:17:38.108980 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e37d7e9-a64a-48c7-9887-bf069dc53b95 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:38.131072 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a254f6-66e9-4eb0-a7b5-d65492cd3159 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:38.145516 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:17:38.161240 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783184dd-8c48-4c9d-bccb-f5b0c4aa4a75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:38.161916 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783184dd-8c48-4c9d-bccb-f5b0c4aa4a75 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:38.162196 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783184dd-8c48-4c9d-bccb-f5b0c4aa4a75 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:38.162464 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783184dd-8c48-4c9d-bccb-f5b0c4aa4a75 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:38.169113 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe28cbe6-9dbd-4a69-a210-81154759aee7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:38.169706 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783184dd-8c48-4c9d-bccb-f5b0c4aa4a75 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:17:38.170058 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe28cbe6-9dbd-4a69-a210-81154759aee7 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:38.170441 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe28cbe6-9dbd-4a69-a210-81154759aee7 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:38.170836 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe28cbe6-9dbd-4a69-a210-81154759aee7 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:38.171170 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783184dd-8c48-4c9d-bccb-f5b0c4aa4a75 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:17:38.181682 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe28cbe6-9dbd-4a69-a210-81154759aee7 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:17:38.183457 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe28cbe6-9dbd-4a69-a210-81154759aee7 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:17:38.186750 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-783184dd-8c48-4c9d-bccb-f5b0c4aa4a75 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 04:17:38.186750 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 04:17:38.186750 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 04:17:38.186750 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 04:17:38.186750 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 04:17:38.186750 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 04:17:38.186750 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 04:17:38.186750 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 04:17:38.186750 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 04:17:38.186750 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 04:17:38.186750 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 04:17:38.186750 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 04:17:38.186750 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 06 04:17:38.186750 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 06 04:17:38.186750 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 06 04:17:38.187646 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 06 04:17:38.187646 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 04:17:38.187646 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 04:17:38.187646 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 04:17:38.187646 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 04:17:38.187646 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 04:17:38.187646 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 04:17:38.187646 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 04:17:38.187646 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 04:17:38.187646 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 04:17:38.187646 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 04:17:38.187646 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 04:17:38.187646 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 04:17:38.187646 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 04:17:38.187646 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 04:17:38.187646 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 04:17:38.187646 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 04:17:38.188541 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 06 04:17:38.188541 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 06 04:17:38.188541 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 06 04:17:38.188541 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 06 04:17:38.188541 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 06 04:17:38.188541 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 06 04:17:38.188541 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 04:17:38.188541 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 04:17:38.188541 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 293/570] 10.222.0.30 () {56 vars in 1111 bytes} [Tue Sep 6 04:17:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 04:17:38.189028 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783184dd-8c48-4c9d-bccb-f5b0c4aa4a75 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:38.202795 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-fe28cbe6-9dbd-4a69-a210-81154759aee7 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 06 04:17:38.202795 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 04:17:38.202795 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 04:17:38.202795 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 04:17:38.202795 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 04:17:38.202795 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 04:17:38.202795 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 04:17:38.202795 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 04:17:38.202795 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 04:17:38.202795 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 04:17:38.202795 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 04:17:38.202795 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 04:17:38.202795 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 06 04:17:38.202795 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 06 04:17:38.202795 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 06 04:17:38.202795 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 06 04:17:38.204906 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 04:17:38.204906 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 04:17:38.204906 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 06 04:17:38.204906 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 04:17:38.204906 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 06 04:17:38.204906 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 04:17:38.204906 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 04:17:38.204906 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 04:17:38.204906 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 04:17:38.204906 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 04:17:38.204906 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 04:17:38.204906 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 04:17:38.204906 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 04:17:38.204906 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 04:17:38.204906 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 04:17:38.204906 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 04:17:38.204906 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 04:17:38.204906 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 04:17:38.206041 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 04:17:38.206041 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 04:17:38.206041 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 06 04:17:38.206041 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 06 04:17:38.206041 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 06 04:17:38.206041 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 06 04:17:38.206041 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 06 04:17:38.206041 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 06 04:17:38.206041 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 06 04:17:38.206041 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 04:17:38.209530 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 278/571] 10.222.0.30 () {56 vars in 1099 bytes} [Tue Sep 6 04:17:37 2022] GET /identity/v3/users/glance => generated 84 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 04:17:38.210718 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4888baf3-8930-4440-a691-11d708211fec None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:38.211013 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4888baf3-8930-4440-a691-11d708211fec None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:38.211296 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4888baf3-8930-4440-a691-11d708211fec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:38.234717 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-4888baf3-8930-4440-a691-11d708211fec None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:17:38.235930 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-4888baf3-8930-4440-a691-11d708211fec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:17:38.321582 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 279/572] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Sep 6 04:17:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 04:17:38.322591 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4888baf3-8930-4440-a691-11d708211fec None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:38.335114 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783184dd-8c48-4c9d-bccb-f5b0c4aa4a75 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:38.377865 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a18e61-5b35-4036-8758-56e13e1b8ad5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:38.381626 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a18e61-5b35-4036-8758-56e13e1b8ad5 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:38.381626 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a18e61-5b35-4036-8758-56e13e1b8ad5 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:38.381626 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a18e61-5b35-4036-8758-56e13e1b8ad5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:38.391623 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a18e61-5b35-4036-8758-56e13e1b8ad5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:17:38.393414 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a18e61-5b35-4036-8758-56e13e1b8ad5 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:17:38.402611 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:17:38.411385 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-81a18e61-5b35-4036-8758-56e13e1b8ad5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 04:17:38.411385 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 04:17:38.411385 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 04:17:38.411385 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 04:17:38.411385 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 04:17:38.411385 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 04:17:38.411385 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 04:17:38.411385 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 04:17:38.411385 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 04:17:38.411385 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 04:17:38.411385 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 04:17:38.411385 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 04:17:38.411385 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 04:17:38.411385 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 04:17:38.411385 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 04:17:38.411385 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 04:17:38.419034 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 04:17:38.419034 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 04:17:38.419034 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 04:17:38.419034 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 04:17:38.419034 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 04:17:38.419034 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 04:17:38.419034 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 04:17:38.419034 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 04:17:38.419034 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 04:17:38.419034 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 04:17:38.419034 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 04:17:38.419034 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 04:17:38.419034 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 04:17:38.419034 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 04:17:38.419034 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 04:17:38.419034 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 04:17:38.419034 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 04:17:38.419954 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 04:17:38.419954 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 04:17:38.419954 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 04:17:38.419954 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 294/573] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Sep 6 04:17:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 04:17:38.419954 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a18e61-5b35-4036-8758-56e13e1b8ad5 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:38.507873 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4888baf3-8930-4440-a691-11d708211fec None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:38.509677 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:17:38.551577 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b804b99-7d45-4ca8-ab93-70d20f136e01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:38.552211 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b804b99-7d45-4ca8-ab93-70d20f136e01 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:38.552490 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b804b99-7d45-4ca8-ab93-70d20f136e01 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:38.552766 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b804b99-7d45-4ca8-ab93-70d20f136e01 None admin] PATH_INFO: `/v3/projects` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:38.553523 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b804b99-7d45-4ca8-ab93-70d20f136e01 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:17:38.553828 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b804b99-7d45-4ca8-ab93-70d20f136e01 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:17:38.554893 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a18e61-5b35-4036-8758-56e13e1b8ad5 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:38.555270 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b804b99-7d45-4ca8-ab93-70d20f136e01 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:17:38.569976 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 280/574] 10.222.0.30 () {56 vars in 1148 bytes} [Tue Sep 6 04:17:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:17:38.570668 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b804b99-7d45-4ca8-ab93-70d20f136e01 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:38.611515 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180b84e3-016c-45bd-8455-41eb58f26590 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:38.612371 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180b84e3-016c-45bd-8455-41eb58f26590 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:38.612371 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180b84e3-016c-45bd-8455-41eb58f26590 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:38.612701 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180b84e3-016c-45bd-8455-41eb58f26590 None admin] PATH_INFO: `/v3/users` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:38.614139 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180b84e3-016c-45bd-8455-41eb58f26590 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:17:38.614445 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180b84e3-016c-45bd-8455-41eb58f26590 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:17:38.617101 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180b84e3-016c-45bd-8455-41eb58f26590 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:17:38.641240 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 295/575] 10.222.0.30 () {56 vars in 1137 bytes} [Tue Sep 6 04:17:38 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:17:38.641240 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180b84e3-016c-45bd-8455-41eb58f26590 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:38.701984 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b804b99-7d45-4ca8-ab93-70d20f136e01 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:38.703623 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:17:38.744357 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32d54b1-12cc-4dfa-a7e9-b4299267bc61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:38.744995 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32d54b1-12cc-4dfa-a7e9-b4299267bc61 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:38.745274 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32d54b1-12cc-4dfa-a7e9-b4299267bc61 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:38.745580 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32d54b1-12cc-4dfa-a7e9-b4299267bc61 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:38.756347 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180b84e3-016c-45bd-8455-41eb58f26590 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:38.759833 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a32d54b1-12cc-4dfa-a7e9-b4299267bc61 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 04:17:38.765688 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-a32d54b1-12cc-4dfa-a7e9-b4299267bc61 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 06 04:17:38.765688 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 04:17:38.765688 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 04:17:38.765688 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 04:17:38.765688 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 04:17:38.765688 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 04:17:38.765688 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 04:17:38.765688 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 04:17:38.765688 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 04:17:38.765688 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 04:17:38.765688 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 04:17:38.765688 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 04:17:38.765688 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 04:17:38.765688 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 04:17:38.765688 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 04:17:38.765688 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 04:17:38.766638 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 04:17:38.766638 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 04:17:38.766638 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 04:17:38.766638 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 04:17:38.766638 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 04:17:38.766638 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 04:17:38.766638 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 04:17:38.766638 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 04:17:38.766638 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 04:17:38.766638 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 04:17:38.766638 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 04:17:38.766638 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 04:17:38.766638 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 04:17:38.766638 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 04:17:38.766638 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 04:17:38.766638 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 04:17:38.768356 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 04:17:38.768356 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 04:17:38.768356 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 04:17:38.768356 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 04:17:38.768356 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 04:17:38.768356 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 04:17:38.768356 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 04:17:38.768356 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 04:17:38.768356 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 04:17:38.768356 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 04:17:38.768356 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 06 04:17:38.768356 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 04:17:38.768356 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 281/576] 10.222.0.30 () {56 vars in 1102 bytes} [Tue Sep 6 04:17:38 2022] GET /identity/v3/roles/service => generated 85 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 04:17:38.768356 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32d54b1-12cc-4dfa-a7e9-b4299267bc61 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:38.797463 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5877c0c4-240f-418b-92fb-f6963e181894 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:38.798080 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5877c0c4-240f-418b-92fb-f6963e181894 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:38.798365 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5877c0c4-240f-418b-92fb-f6963e181894 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:38.798638 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5877c0c4-240f-418b-92fb-f6963e181894 None admin] PATH_INFO: `/v3/domains` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:38.799281 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5877c0c4-240f-418b-92fb-f6963e181894 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:17:38.799583 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5877c0c4-240f-418b-92fb-f6963e181894 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:17:38.802103 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5877c0c4-240f-418b-92fb-f6963e181894 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:17:38.802965 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:17:38.814288 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 296/577] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Sep 6 04:17:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:17:38.815166 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5877c0c4-240f-418b-92fb-f6963e181894 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:38.900362 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32d54b1-12cc-4dfa-a7e9-b4299267bc61 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:38.932989 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5877c0c4-240f-418b-92fb-f6963e181894 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:38.934832 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9da424-e2b9-4a06-8ba9-0169eb6f6085 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:38.935440 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9da424-e2b9-4a06-8ba9-0169eb6f6085 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:38.935715 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9da424-e2b9-4a06-8ba9-0169eb6f6085 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:38.935984 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9da424-e2b9-4a06-8ba9-0169eb6f6085 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:38.936702 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9da424-e2b9-4a06-8ba9-0169eb6f6085 None admin] RBAC: Adding query filter params (role.id=83cf0a194f8d45e792411e2d9afdfe50, scope.project.id=b707733ce65c42edbc1fd95ad0a1c06c, user.id=a676ec7f92a144edbeca8e6029ba9ee6) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:17:38.937005 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9da424-e2b9-4a06-8ba9-0169eb6f6085 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:17:38.938466 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9da424-e2b9-4a06-8ba9-0169eb6f6085 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:17:38.944271 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 282/578] 10.222.0.30 () {56 vars in 1374 bytes} [Tue Sep 6 04:17:38 2022] GET /identity/v3/role_assignments?user.id=a676ec7f92a144edbeca8e6029ba9ee6&scope.project.id=b707733ce65c42edbc1fd95ad0a1c06c&role.id=83cf0a194f8d45e792411e2d9afdfe50 => generated 259 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:17:38.945063 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9da424-e2b9-4a06-8ba9-0169eb6f6085 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:38.958800 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:17:38.980746 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ababcb-2436-49f0-a73d-439a7b47dda6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:38.981434 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ababcb-2436-49f0-a73d-439a7b47dda6 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:38.981761 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ababcb-2436-49f0-a73d-439a7b47dda6 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:38.981986 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ababcb-2436-49f0-a73d-439a7b47dda6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:38.995124 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ababcb-2436-49f0-a73d-439a7b47dda6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:17:38.996647 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ababcb-2436-49f0-a73d-439a7b47dda6 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:17:39.012856 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-75ababcb-2436-49f0-a73d-439a7b47dda6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 04:17:39.012856 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 04:17:39.012856 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 04:17:39.012856 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 04:17:39.012856 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 04:17:39.012856 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 04:17:39.012856 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 04:17:39.012856 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 04:17:39.012856 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 04:17:39.012856 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 04:17:39.012856 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 04:17:39.012856 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 04:17:39.012856 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 04:17:39.012856 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 04:17:39.012856 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 04:17:39.012856 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 04:17:39.014094 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 04:17:39.014094 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 04:17:39.014094 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 04:17:39.014094 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 04:17:39.014094 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 04:17:39.014094 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 04:17:39.014094 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 04:17:39.014094 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 04:17:39.014094 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 04:17:39.014094 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 04:17:39.014094 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 04:17:39.014094 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 04:17:39.014094 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 04:17:39.014094 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 04:17:39.014094 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 04:17:39.014094 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 04:17:39.014094 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 04:17:39.015020 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 04:17:39.015020 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 04:17:39.015020 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 04:17:39.015020 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 297/579] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Sep 6 04:17:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 04:17:39.015256 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ababcb-2436-49f0-a73d-439a7b47dda6 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:39.051660 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:17:39.068667 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9da424-e2b9-4a06-8ba9-0169eb6f6085 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:39.111172 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58fb949-a2a4-476f-916c-7fe5c8b4694c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:39.111817 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58fb949-a2a4-476f-916c-7fe5c8b4694c None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:39.112095 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58fb949-a2a4-476f-916c-7fe5c8b4694c None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:39.112365 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58fb949-a2a4-476f-916c-7fe5c8b4694c None admin] PATH_INFO: `/v3/roles/service` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:39.122936 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58fb949-a2a4-476f-916c-7fe5c8b4694c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 04:17:39.128141 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-a58fb949-a2a4-476f-916c-7fe5c8b4694c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 06 04:17:39.128141 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 04:17:39.128141 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 04:17:39.128141 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 04:17:39.128141 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 04:17:39.128141 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 04:17:39.128141 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 04:17:39.128141 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 04:17:39.128141 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 04:17:39.128141 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 04:17:39.128141 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 04:17:39.128141 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 04:17:39.128141 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 04:17:39.128141 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 04:17:39.128141 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 04:17:39.128141 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 04:17:39.129120 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 04:17:39.129120 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 04:17:39.129120 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 04:17:39.129120 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 04:17:39.129120 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 04:17:39.129120 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 04:17:39.129120 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 04:17:39.129120 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 04:17:39.129120 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 04:17:39.129120 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 04:17:39.129120 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 04:17:39.129120 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 04:17:39.129120 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 04:17:39.129120 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 04:17:39.129120 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 04:17:39.129120 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 04:17:39.130028 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 04:17:39.130028 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 04:17:39.130028 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 04:17:39.130028 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 04:17:39.130028 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 04:17:39.130028 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 04:17:39.130028 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 04:17:39.130028 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 04:17:39.130028 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 04:17:39.130028 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 04:17:39.130028 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 06 04:17:39.130028 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 04:17:39.131063 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ababcb-2436-49f0-a73d-439a7b47dda6 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:39.134207 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 283/580] 10.222.0.30 () {56 vars in 1102 bytes} [Tue Sep 6 04:17:38 2022] GET /identity/v3/roles/service => generated 85 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 04:17:39.134815 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58fb949-a2a4-476f-916c-7fe5c8b4694c None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:39.172250 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea65d14-8085-406a-b80d-2215ad72bd7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:39.172783 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea65d14-8085-406a-b80d-2215ad72bd7a None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:39.173059 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea65d14-8085-406a-b80d-2215ad72bd7a None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:39.173348 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea65d14-8085-406a-b80d-2215ad72bd7a None admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:39.174024 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea65d14-8085-406a-b80d-2215ad72bd7a None admin] RBAC: Adding query filter params (name=service) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:17:39.174319 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea65d14-8085-406a-b80d-2215ad72bd7a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:17:39.176223 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea65d14-8085-406a-b80d-2215ad72bd7a None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:17:39.191698 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 298/581] 10.222.0.30 () {56 vars in 1103 bytes} [Tue Sep 6 04:17:39 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:17:39.191698 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea65d14-8085-406a-b80d-2215ad72bd7a None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:39.230128 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:17:39.238562 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58fb949-a2a4-476f-916c-7fe5c8b4694c None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:39.279279 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb79204-05e5-4173-9326-c72dffa71cf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:39.279913 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb79204-05e5-4173-9326-c72dffa71cf9 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:39.280188 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb79204-05e5-4173-9326-c72dffa71cf9 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:39.280570 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb79204-05e5-4173-9326-c72dffa71cf9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:39.288127 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb79204-05e5-4173-9326-c72dffa71cf9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:17:39.289699 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb79204-05e5-4173-9326-c72dffa71cf9 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:17:39.302226 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-abb79204-05e5-4173-9326-c72dffa71cf9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 04:17:39.302226 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 04:17:39.302226 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 04:17:39.302226 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 04:17:39.302226 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 04:17:39.302226 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 04:17:39.302226 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 04:17:39.302226 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 04:17:39.302226 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 04:17:39.302226 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 04:17:39.302226 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 04:17:39.302226 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 04:17:39.302226 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 06 04:17:39.302226 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 06 04:17:39.302226 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 06 04:17:39.303133 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 06 04:17:39.303133 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 04:17:39.303133 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 04:17:39.303133 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 04:17:39.303133 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 04:17:39.303133 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 04:17:39.303133 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 04:17:39.303133 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 04:17:39.303133 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 04:17:39.303133 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 04:17:39.303133 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 04:17:39.303133 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 04:17:39.303133 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 04:17:39.303133 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 04:17:39.303133 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 04:17:39.303133 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 04:17:39.303133 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 04:17:39.304039 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 06 04:17:39.304039 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 06 04:17:39.304039 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 06 04:17:39.304039 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 06 04:17:39.304039 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 06 04:17:39.304039 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 06 04:17:39.304039 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 04:17:39.304039 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 04:17:39.304484 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea65d14-8085-406a-b80d-2215ad72bd7a None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:39.308290 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 284/582] 10.222.0.30 () {56 vars in 1111 bytes} [Tue Sep 6 04:17:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 04:17:39.309194 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb79204-05e5-4173-9326-c72dffa71cf9 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:39.319614 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:17:39.341175 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe34b714-a17f-457e-b00c-9651e91457c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:39.341814 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe34b714-a17f-457e-b00c-9651e91457c2 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:39.342091 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe34b714-a17f-457e-b00c-9651e91457c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:39.342359 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe34b714-a17f-457e-b00c-9651e91457c2 None admin] PATH_INFO: `/v3/domains` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:39.343191 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe34b714-a17f-457e-b00c-9651e91457c2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:17:39.343753 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe34b714-a17f-457e-b00c-9651e91457c2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:17:39.345140 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe34b714-a17f-457e-b00c-9651e91457c2 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:17:39.356902 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 299/583] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Sep 6 04:17:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:17:39.357746 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe34b714-a17f-457e-b00c-9651e91457c2 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:39.427479 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb79204-05e5-4173-9326-c72dffa71cf9 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:39.461888 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d26c415-1df0-48e4-8039-50c44d0032da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:39.462500 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d26c415-1df0-48e4-8039-50c44d0032da None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:39.462783 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d26c415-1df0-48e4-8039-50c44d0032da None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:39.463058 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d26c415-1df0-48e4-8039-50c44d0032da None admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:39.463721 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d26c415-1df0-48e4-8039-50c44d0032da None admin] RBAC: Adding query filter params (name=service) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:17:39.464024 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d26c415-1df0-48e4-8039-50c44d0032da None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:17:39.465480 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d26c415-1df0-48e4-8039-50c44d0032da None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:17:39.474313 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 285/584] 10.222.0.30 () {56 vars in 1103 bytes} [Tue Sep 6 04:17:39 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:17:39.475277 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d26c415-1df0-48e4-8039-50c44d0032da None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:39.500184 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:17:39.513114 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe34b714-a17f-457e-b00c-9651e91457c2 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:39.546682 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:17:39.550192 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6514bf39-33d8-4557-8c2a-1201c7f85570 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:39.551451 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6514bf39-33d8-4557-8c2a-1201c7f85570 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:39.551904 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6514bf39-33d8-4557-8c2a-1201c7f85570 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:39.552220 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6514bf39-33d8-4557-8c2a-1201c7f85570 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:39.562976 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6514bf39-33d8-4557-8c2a-1201c7f85570 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:17:39.564538 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6514bf39-33d8-4557-8c2a-1201c7f85570 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:17:39.579671 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-6514bf39-33d8-4557-8c2a-1201c7f85570 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 04:17:39.579671 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 04:17:39.579671 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 04:17:39.579671 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 04:17:39.579671 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 04:17:39.579671 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 04:17:39.579671 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 04:17:39.579671 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 04:17:39.579671 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 04:17:39.579671 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 04:17:39.579671 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 04:17:39.579671 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 04:17:39.579671 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 04:17:39.579671 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 04:17:39.579671 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 04:17:39.579671 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 04:17:39.580654 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 04:17:39.580654 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 04:17:39.580654 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 04:17:39.580654 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 04:17:39.580654 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 04:17:39.580654 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 04:17:39.580654 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 04:17:39.580654 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 04:17:39.580654 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 04:17:39.580654 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 04:17:39.580654 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 04:17:39.580654 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 04:17:39.580654 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 04:17:39.580654 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 04:17:39.580654 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 04:17:39.580654 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 04:17:39.580654 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 04:17:39.582577 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 04:17:39.582577 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 04:17:39.582577 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 04:17:39.582577 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 300/585] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Sep 6 04:17:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 04:17:39.582577 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6514bf39-33d8-4557-8c2a-1201c7f85570 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:39.585592 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d26c415-1df0-48e4-8039-50c44d0032da None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:39.623880 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd35d8e-d277-4371-9a5c-b5a4e3a44a31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:39.624489 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd35d8e-d277-4371-9a5c-b5a4e3a44a31 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:39.624764 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd35d8e-d277-4371-9a5c-b5a4e3a44a31 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:39.625031 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd35d8e-d277-4371-9a5c-b5a4e3a44a31 None admin] PATH_INFO: `/v3/projects` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:39.625741 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd35d8e-d277-4371-9a5c-b5a4e3a44a31 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:17:39.626041 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd35d8e-d277-4371-9a5c-b5a4e3a44a31 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:17:39.627716 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd35d8e-d277-4371-9a5c-b5a4e3a44a31 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:17:39.639970 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 286/586] 10.222.0.30 () {56 vars in 1148 bytes} [Tue Sep 6 04:17:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:17:39.645937 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd35d8e-d277-4371-9a5c-b5a4e3a44a31 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:39.694099 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6514bf39-33d8-4557-8c2a-1201c7f85570 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:39.733673 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c00455-a7d4-4d4d-a840-4da43ee686e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:39.734309 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c00455-a7d4-4d4d-a840-4da43ee686e0 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:39.734569 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c00455-a7d4-4d4d-a840-4da43ee686e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:39.734838 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c00455-a7d4-4d4d-a840-4da43ee686e0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:39.742427 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c00455-a7d4-4d4d-a840-4da43ee686e0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:17:39.743972 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c00455-a7d4-4d4d-a840-4da43ee686e0 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:17:39.754978 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd35d8e-d277-4371-9a5c-b5a4e3a44a31 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:39.756563 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-e3c00455-a7d4-4d4d-a840-4da43ee686e0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 04:17:39.756563 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 04:17:39.756563 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 04:17:39.756563 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 04:17:39.756563 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 04:17:39.756563 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 04:17:39.756563 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 04:17:39.756563 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 04:17:39.756563 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 04:17:39.756563 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 04:17:39.756563 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 04:17:39.756563 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 04:17:39.756563 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 06 04:17:39.756563 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 06 04:17:39.756563 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 06 04:17:39.757750 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 06 04:17:39.757750 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 04:17:39.757750 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 04:17:39.757750 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 04:17:39.757750 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 04:17:39.757750 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 04:17:39.757750 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 04:17:39.757750 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 04:17:39.757750 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 04:17:39.757750 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 04:17:39.757750 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 04:17:39.757750 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 04:17:39.757750 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 04:17:39.757750 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 04:17:39.757750 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 04:17:39.757750 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 04:17:39.757750 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 04:17:39.763410 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 06 04:17:39.763410 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 06 04:17:39.763410 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 06 04:17:39.763410 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 06 04:17:39.763410 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 06 04:17:39.763410 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 06 04:17:39.763410 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 04:17:39.763410 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 04:17:39.763410 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 301/587] 10.222.0.30 () {56 vars in 1111 bytes} [Tue Sep 6 04:17:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 04:17:39.763410 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c00455-a7d4-4d4d-a840-4da43ee686e0 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:39.763410 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:17:39.791518 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:17:39.799643 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b57aa4-69b9-4127-ba71-dc730503d46f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:39.800877 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b57aa4-69b9-4127-ba71-dc730503d46f None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:39.801228 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b57aa4-69b9-4127-ba71-dc730503d46f None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:39.801542 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b57aa4-69b9-4127-ba71-dc730503d46f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:39.811894 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b57aa4-69b9-4127-ba71-dc730503d46f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:17:39.813358 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b57aa4-69b9-4127-ba71-dc730503d46f None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:17:39.828655 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-75b57aa4-69b9-4127-ba71-dc730503d46f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 04:17:39.828655 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 04:17:39.828655 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 04:17:39.828655 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 04:17:39.828655 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 04:17:39.828655 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 04:17:39.828655 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 04:17:39.828655 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 04:17:39.828655 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 04:17:39.828655 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 04:17:39.828655 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 04:17:39.828655 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 04:17:39.828655 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 04:17:39.828655 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 04:17:39.828655 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 04:17:39.828655 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 04:17:39.829848 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 04:17:39.829848 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 04:17:39.829848 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 04:17:39.829848 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 04:17:39.829848 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 04:17:39.829848 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 04:17:39.829848 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 04:17:39.829848 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 04:17:39.829848 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 04:17:39.829848 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 04:17:39.829848 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 04:17:39.829848 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 04:17:39.829848 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 04:17:39.829848 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 04:17:39.829848 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 04:17:39.829848 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 04:17:39.829848 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 04:17:39.831013 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 04:17:39.831013 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 04:17:39.831013 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 04:17:39.832777 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 287/588] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Sep 6 04:17:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 04:17:39.833630 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b57aa4-69b9-4127-ba71-dc730503d46f None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:39.870552 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c00455-a7d4-4d4d-a840-4da43ee686e0 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:39.905340 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe878914-0d12-4693-bb90-950c35ccc3a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:39.905897 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe878914-0d12-4693-bb90-950c35ccc3a6 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:39.906161 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe878914-0d12-4693-bb90-950c35ccc3a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:39.906430 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe878914-0d12-4693-bb90-950c35ccc3a6 None admin] PATH_INFO: `/v3/domains` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:39.907073 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe878914-0d12-4693-bb90-950c35ccc3a6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:17:39.907373 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe878914-0d12-4693-bb90-950c35ccc3a6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:17:39.908734 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe878914-0d12-4693-bb90-950c35ccc3a6 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:17:39.922083 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 302/589] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Sep 6 04:17:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:17:39.923024 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe878914-0d12-4693-bb90-950c35ccc3a6 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:39.949573 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b57aa4-69b9-4127-ba71-dc730503d46f None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:39.987434 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d8367a-35d4-4c10-a24e-8b951bcf1bf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:39.988368 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d8367a-35d4-4c10-a24e-8b951bcf1bf0 None admin] REQUEST_METHOD: `PUT` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:39.988749 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d8367a-35d4-4c10-a24e-8b951bcf1bf0 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:39.989033 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d8367a-35d4-4c10-a24e-8b951bcf1bf0 None admin] PATH_INFO: `/v3/projects/b707733ce65c42edbc1fd95ad0a1c06c/users/9db138b77afa4b068244b8ef715404fe/roles/83cf0a194f8d45e792411e2d9afdfe50` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:39.989774 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:17:40.004917 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d8367a-35d4-4c10-a24e-8b951bcf1bf0 None admin] RBAC: Authorizing `identity:create_grant(project_id=b707733ce65c42edbc1fd95ad0a1c06c, user_id=9db138b77afa4b068244b8ef715404fe, role_id=83cf0a194f8d45e792411e2d9afdfe50)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:17:40.006351 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d8367a-35d4-4c10-a24e-8b951bcf1bf0 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:17:40.034481 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe878914-0d12-4693-bb90-950c35ccc3a6 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:40.068501 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93b6f19-49ef-4b4b-88a9-e2c98f4b922e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:40.069108 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93b6f19-49ef-4b4b-88a9-e2c98f4b922e None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:40.069422 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93b6f19-49ef-4b4b-88a9-e2c98f4b922e None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:40.069696 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93b6f19-49ef-4b4b-88a9-e2c98f4b922e None admin] PATH_INFO: `/v3/projects` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:40.070385 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93b6f19-49ef-4b4b-88a9-e2c98f4b922e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:17:40.070689 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93b6f19-49ef-4b4b-88a9-e2c98f4b922e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:17:40.072141 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93b6f19-49ef-4b4b-88a9-e2c98f4b922e None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:17:40.083520 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 303/590] 10.222.0.30 () {56 vars in 1148 bytes} [Tue Sep 6 04:17:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:17:40.084305 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93b6f19-49ef-4b4b-88a9-e2c98f4b922e None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:40.093351 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 288/591] 10.222.0.30 () {58 vars in 1439 bytes} [Tue Sep 6 04:17:39 2022] PUT /identity/v3/projects/b707733ce65c42edbc1fd95ad0a1c06c/users/9db138b77afa4b068244b8ef715404fe/roles/83cf0a194f8d45e792411e2d9afdfe50 => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 04:17:40.106568 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d8367a-35d4-4c10-a24e-8b951bcf1bf0 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:40.151965 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:17:40.218664 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93b6f19-49ef-4b4b-88a9-e2c98f4b922e None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:40.252476 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:17:40.252670 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d8367a-35d4-4c10-a24e-8b951bcf1bf0 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:40.270151 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6203c7cd-3b17-4217-8cb5-f1fd1ec44772 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:40.270775 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6203c7cd-3b17-4217-8cb5-f1fd1ec44772 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:40.271056 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6203c7cd-3b17-4217-8cb5-f1fd1ec44772 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:40.271326 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6203c7cd-3b17-4217-8cb5-f1fd1ec44772 None admin] PATH_INFO: `/v3/domains` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:40.272003 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6203c7cd-3b17-4217-8cb5-f1fd1ec44772 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:17:40.272304 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6203c7cd-3b17-4217-8cb5-f1fd1ec44772 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:17:40.274144 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6203c7cd-3b17-4217-8cb5-f1fd1ec44772 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:17:40.289337 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 304/592] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Sep 6 04:17:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:17:40.290166 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6203c7cd-3b17-4217-8cb5-f1fd1ec44772 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:40.292930 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29bfc903-283c-4292-b87a-5060ad6b8249 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:40.294741 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bfc903-283c-4292-b87a-5060ad6b8249 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:40.295217 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bfc903-283c-4292-b87a-5060ad6b8249 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:40.295555 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bfc903-283c-4292-b87a-5060ad6b8249 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:40.303857 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29bfc903-283c-4292-b87a-5060ad6b8249 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:17:40.305369 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29bfc903-283c-4292-b87a-5060ad6b8249 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:17:40.317739 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-29bfc903-283c-4292-b87a-5060ad6b8249 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 06 04:17:40.317739 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 04:17:40.317739 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 04:17:40.317739 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 04:17:40.317739 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 04:17:40.317739 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 04:17:40.317739 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 04:17:40.317739 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 04:17:40.317739 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 04:17:40.317739 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 04:17:40.317739 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 04:17:40.317739 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 04:17:40.317739 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 06 04:17:40.317739 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 06 04:17:40.317739 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 06 04:17:40.317739 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 06 04:17:40.318859 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 04:17:40.318859 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 04:17:40.318859 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 06 04:17:40.318859 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 04:17:40.318859 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 06 04:17:40.318859 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 04:17:40.318859 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 04:17:40.318859 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 04:17:40.318859 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 04:17:40.318859 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 04:17:40.318859 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 04:17:40.318859 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 04:17:40.318859 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 04:17:40.318859 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 04:17:40.318859 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 04:17:40.318859 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 04:17:40.318859 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 04:17:40.318859 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 04:17:40.319860 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 04:17:40.319860 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 04:17:40.319860 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 06 04:17:40.319860 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 06 04:17:40.319860 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 06 04:17:40.319860 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 06 04:17:40.319860 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 06 04:17:40.319860 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 06 04:17:40.319860 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 06 04:17:40.319860 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 04:17:40.319860 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 289/593] 10.222.0.30 () {56 vars in 1099 bytes} [Tue Sep 6 04:17:40 2022] GET /identity/v3/users/cinder => generated 84 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 04:17:40.320446 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29bfc903-283c-4292-b87a-5060ad6b8249 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:40.416133 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6203c7cd-3b17-4217-8cb5-f1fd1ec44772 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:40.454203 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:17:40.458958 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41cc0e76-cec2-4f7c-9167-f196bea61aee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:40.459725 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cc0e76-cec2-4f7c-9167-f196bea61aee None admin] REQUEST_METHOD: `PUT` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:40.460064 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cc0e76-cec2-4f7c-9167-f196bea61aee None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:40.460402 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cc0e76-cec2-4f7c-9167-f196bea61aee None admin] PATH_INFO: `/v3/projects/b707733ce65c42edbc1fd95ad0a1c06c/users/ae5c31fb1d0342c0937bbc83a415113d/roles/83cf0a194f8d45e792411e2d9afdfe50` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:40.466697 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29bfc903-283c-4292-b87a-5060ad6b8249 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:40.478122 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41cc0e76-cec2-4f7c-9167-f196bea61aee None admin] RBAC: Authorizing `identity:create_grant(project_id=b707733ce65c42edbc1fd95ad0a1c06c, user_id=ae5c31fb1d0342c0937bbc83a415113d, role_id=83cf0a194f8d45e792411e2d9afdfe50)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:17:40.479505 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41cc0e76-cec2-4f7c-9167-f196bea61aee None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:17:40.510368 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c90939b-e11e-4a65-8152-bf568f63b7bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:40.511247 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c90939b-e11e-4a65-8152-bf568f63b7bb None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:40.511677 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c90939b-e11e-4a65-8152-bf568f63b7bb None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:40.512098 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c90939b-e11e-4a65-8152-bf568f63b7bb None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:40.521069 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c90939b-e11e-4a65-8152-bf568f63b7bb None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:17:40.522710 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c90939b-e11e-4a65-8152-bf568f63b7bb None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:17:40.534610 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-6c90939b-e11e-4a65-8152-bf568f63b7bb None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 06 04:17:40.534610 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 04:17:40.534610 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 04:17:40.534610 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 04:17:40.534610 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 04:17:40.534610 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 04:17:40.534610 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 04:17:40.534610 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 04:17:40.534610 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 04:17:40.534610 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 04:17:40.534610 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 04:17:40.534610 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 04:17:40.534610 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 06 04:17:40.534610 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 06 04:17:40.534610 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 06 04:17:40.534610 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 06 04:17:40.536150 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 04:17:40.536150 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 04:17:40.536150 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 06 04:17:40.536150 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 04:17:40.536150 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 06 04:17:40.536150 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 04:17:40.536150 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 04:17:40.536150 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 04:17:40.536150 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 04:17:40.536150 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 04:17:40.536150 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 04:17:40.536150 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 04:17:40.536150 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 04:17:40.536150 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 04:17:40.536150 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 04:17:40.536150 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 04:17:40.536150 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 04:17:40.536150 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 04:17:40.537071 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 04:17:40.537071 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 04:17:40.537071 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 06 04:17:40.537071 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 06 04:17:40.537071 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 06 04:17:40.537071 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 06 04:17:40.537071 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 06 04:17:40.537071 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 06 04:17:40.537071 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 06 04:17:40.537071 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 04:17:40.541222 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 290/594] 10.222.0.30 () {56 vars in 1102 bytes} [Tue Sep 6 04:17:40 2022] GET /identity/v3/users/neutron => generated 85 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 04:17:40.542179 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c90939b-e11e-4a65-8152-bf568f63b7bb None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:40.553557 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 305/595] 10.222.0.30 () {58 vars in 1439 bytes} [Tue Sep 6 04:17:40 2022] PUT /identity/v3/projects/b707733ce65c42edbc1fd95ad0a1c06c/users/ae5c31fb1d0342c0937bbc83a415113d/roles/83cf0a194f8d45e792411e2d9afdfe50 => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 04:17:40.554828 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41cc0e76-cec2-4f7c-9167-f196bea61aee None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:40.645909 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:17:40.667131 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c90939b-e11e-4a65-8152-bf568f63b7bb None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:40.713151 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f431330-ebbc-4587-9d89-99e461375104 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:40.714816 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f431330-ebbc-4587-9d89-99e461375104 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:40.715399 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f431330-ebbc-4587-9d89-99e461375104 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:40.717041 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f431330-ebbc-4587-9d89-99e461375104 None admin] PATH_INFO: `/v3/users` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:40.718188 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f431330-ebbc-4587-9d89-99e461375104 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:17:40.719333 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f431330-ebbc-4587-9d89-99e461375104 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:17:40.721777 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f431330-ebbc-4587-9d89-99e461375104 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:17:40.754310 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 291/596] 10.222.0.30 () {56 vars in 1137 bytes} [Tue Sep 6 04:17:40 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:17:40.756081 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f431330-ebbc-4587-9d89-99e461375104 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:40.768669 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:17:40.792290 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41cc0e76-cec2-4f7c-9167-f196bea61aee None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:40.841862 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f619de31-becd-4810-91b2-b0dd86b5e747 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:40.842718 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f619de31-becd-4810-91b2-b0dd86b5e747 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:40.843128 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f619de31-becd-4810-91b2-b0dd86b5e747 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:40.843527 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f619de31-becd-4810-91b2-b0dd86b5e747 None admin] PATH_INFO: `/v3/users` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:40.845379 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f619de31-becd-4810-91b2-b0dd86b5e747 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:17:40.845844 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f619de31-becd-4810-91b2-b0dd86b5e747 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:17:40.848136 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f619de31-becd-4810-91b2-b0dd86b5e747 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:17:40.874943 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 306/597] 10.222.0.30 () {56 vars in 1139 bytes} [Tue Sep 6 04:17:40 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 320 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:17:40.881607 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f619de31-becd-4810-91b2-b0dd86b5e747 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:40.910937 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f431330-ebbc-4587-9d89-99e461375104 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:40.998767 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdae5615-cf93-4659-90dd-db3d9f7c18e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:41.000590 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdae5615-cf93-4659-90dd-db3d9f7c18e9 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:41.002315 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdae5615-cf93-4659-90dd-db3d9f7c18e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:41.002692 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdae5615-cf93-4659-90dd-db3d9f7c18e9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:41.013746 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:17:41.025688 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdae5615-cf93-4659-90dd-db3d9f7c18e9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:17:41.027302 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdae5615-cf93-4659-90dd-db3d9f7c18e9 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:17:41.029278 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f619de31-becd-4810-91b2-b0dd86b5e747 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:41.049155 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-cdae5615-cf93-4659-90dd-db3d9f7c18e9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 04:17:41.049155 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 04:17:41.049155 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 04:17:41.049155 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 04:17:41.049155 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 04:17:41.049155 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 04:17:41.049155 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 04:17:41.049155 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 04:17:41.049155 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 04:17:41.049155 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 04:17:41.049155 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 04:17:41.049155 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 04:17:41.049155 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 04:17:41.049155 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 04:17:41.049155 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 04:17:41.049155 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 04:17:41.050396 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 04:17:41.050396 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 04:17:41.050396 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 04:17:41.050396 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 04:17:41.050396 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 04:17:41.050396 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 04:17:41.050396 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 04:17:41.050396 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 04:17:41.050396 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 04:17:41.050396 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 04:17:41.050396 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 04:17:41.050396 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 04:17:41.050396 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 04:17:41.050396 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 04:17:41.050396 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 04:17:41.050396 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 04:17:41.050396 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 04:17:41.051496 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 04:17:41.051496 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 04:17:41.051496 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 04:17:41.053039 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 292/598] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Sep 6 04:17:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 297 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 04:17:41.054849 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce320a6-bfce-4afe-a44d-bf1b507ca93a None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:41.055683 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce320a6-bfce-4afe-a44d-bf1b507ca93a None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:41.056059 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce320a6-bfce-4afe-a44d-bf1b507ca93a None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:41.058194 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 293/599] 10.222.0.30 () {54 vars in 927 bytes} [Tue Sep 6 04:17:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 04:17:41.058977 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce320a6-bfce-4afe-a44d-bf1b507ca93a None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:41.074383 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce29d00-2164-4aea-8215-a0baeaec6c96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:41.075241 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce29d00-2164-4aea-8215-a0baeaec6c96 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:41.075663 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce29d00-2164-4aea-8215-a0baeaec6c96 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:41.076104 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce29d00-2164-4aea-8215-a0baeaec6c96 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:41.087326 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce29d00-2164-4aea-8215-a0baeaec6c96 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:17:41.088923 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce29d00-2164-4aea-8215-a0baeaec6c96 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:17:41.105633 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-8ce29d00-2164-4aea-8215-a0baeaec6c96 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 04:17:41.105633 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 04:17:41.105633 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 04:17:41.105633 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 04:17:41.105633 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 04:17:41.105633 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 04:17:41.105633 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 04:17:41.105633 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 04:17:41.105633 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 04:17:41.105633 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 04:17:41.105633 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 04:17:41.105633 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 04:17:41.105633 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 04:17:41.105633 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 04:17:41.105633 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 04:17:41.105633 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 04:17:41.106835 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 04:17:41.106835 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 04:17:41.106835 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 04:17:41.106835 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 04:17:41.106835 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 04:17:41.106835 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 04:17:41.106835 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 04:17:41.106835 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 04:17:41.106835 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 04:17:41.106835 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 04:17:41.106835 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 04:17:41.106835 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 04:17:41.106835 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 04:17:41.106835 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 04:17:41.106835 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 04:17:41.106835 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 04:17:41.106835 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 04:17:41.107922 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 04:17:41.107922 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 04:17:41.107922 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 04:17:41.109694 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 307/600] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Sep 6 04:17:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 04:17:41.111095 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e49a90-1aa8-465f-a2f4-1362a85bfd72 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:41.111525 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e49a90-1aa8-465f-a2f4-1362a85bfd72 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:41.111933 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e49a90-1aa8-465f-a2f4-1362a85bfd72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:41.131683 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-21e49a90-1aa8-465f-a2f4-1362a85bfd72 None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:17:41.133162 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-21e49a90-1aa8-465f-a2f4-1362a85bfd72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:17:41.138584 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:17:41.194894 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce320a6-bfce-4afe-a44d-bf1b507ca93a None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:41.214556 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 308/601] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Sep 6 04:17:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 04:17:41.215850 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e49a90-1aa8-465f-a2f4-1362a85bfd72 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:41.237554 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab625fa6-698f-4a2b-b7b0-efecbc67296e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:41.238390 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab625fa6-698f-4a2b-b7b0-efecbc67296e None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:41.238777 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab625fa6-698f-4a2b-b7b0-efecbc67296e None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:41.239142 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab625fa6-698f-4a2b-b7b0-efecbc67296e None admin] PATH_INFO: `/v3/domains` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:41.239892 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab625fa6-698f-4a2b-b7b0-efecbc67296e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:17:41.240294 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab625fa6-698f-4a2b-b7b0-efecbc67296e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:17:41.241787 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab625fa6-698f-4a2b-b7b0-efecbc67296e None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:17:41.259038 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 294/602] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Sep 6 04:17:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:17:41.261463 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab625fa6-698f-4a2b-b7b0-efecbc67296e None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:41.302405 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:17:41.336589 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e49a90-1aa8-465f-a2f4-1362a85bfd72 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:41.367638 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab625fa6-698f-4a2b-b7b0-efecbc67296e None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:41.375819 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c4b4fe-3532-4c34-869d-3d2173825558 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:41.376601 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c4b4fe-3532-4c34-869d-3d2173825558 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:41.377331 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c4b4fe-3532-4c34-869d-3d2173825558 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:41.377766 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c4b4fe-3532-4c34-869d-3d2173825558 None admin] PATH_INFO: `/v3/domains` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:41.378974 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c4b4fe-3532-4c34-869d-3d2173825558 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:17:41.379423 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c4b4fe-3532-4c34-869d-3d2173825558 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:17:41.381032 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c4b4fe-3532-4c34-869d-3d2173825558 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:17:41.395476 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 309/603] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Sep 6 04:17:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:17:41.402153 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:17:41.403889 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c4b4fe-3532-4c34-869d-3d2173825558 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:41.429586 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2518ebe3-8776-4602-8699-42994bdcd519 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:41.430228 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2518ebe3-8776-4602-8699-42994bdcd519 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:41.430507 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2518ebe3-8776-4602-8699-42994bdcd519 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:41.430781 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2518ebe3-8776-4602-8699-42994bdcd519 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:41.443069 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2518ebe3-8776-4602-8699-42994bdcd519 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:17:41.445012 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2518ebe3-8776-4602-8699-42994bdcd519 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:17:41.457247 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-2518ebe3-8776-4602-8699-42994bdcd519 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 04:17:41.457247 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 04:17:41.457247 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 04:17:41.457247 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 04:17:41.457247 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 04:17:41.457247 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 04:17:41.457247 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 04:17:41.457247 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 04:17:41.457247 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 04:17:41.457247 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 04:17:41.457247 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 04:17:41.457247 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 04:17:41.457247 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 06 04:17:41.457247 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 06 04:17:41.457247 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 06 04:17:41.461354 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 06 04:17:41.461354 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 04:17:41.461354 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 04:17:41.461354 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 04:17:41.461354 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 04:17:41.461354 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 04:17:41.461354 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 04:17:41.461354 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 04:17:41.461354 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 04:17:41.461354 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 04:17:41.461354 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 04:17:41.461354 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 04:17:41.461354 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 04:17:41.461354 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 04:17:41.461354 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 04:17:41.461354 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 04:17:41.461354 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 04:17:41.462252 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 06 04:17:41.462252 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 06 04:17:41.462252 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 06 04:17:41.462252 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 06 04:17:41.462252 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 06 04:17:41.462252 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 06 04:17:41.462252 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 04:17:41.462252 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 04:17:41.462252 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 295/604] 10.222.0.30 () {56 vars in 1111 bytes} [Tue Sep 6 04:17:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 04:17:41.470384 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2518ebe3-8776-4602-8699-42994bdcd519 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:41.615394 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2518ebe3-8776-4602-8699-42994bdcd519 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:41.626071 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c4b4fe-3532-4c34-869d-3d2173825558 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:41.650492 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d737306-42c8-49c5-8737-3bc7a09d55ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:41.651114 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d737306-42c8-49c5-8737-3bc7a09d55ba None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:41.651397 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d737306-42c8-49c5-8737-3bc7a09d55ba None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:41.651719 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d737306-42c8-49c5-8737-3bc7a09d55ba None admin] PATH_INFO: `/v3/projects` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:41.652358 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d737306-42c8-49c5-8737-3bc7a09d55ba None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:17:41.652663 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d737306-42c8-49c5-8737-3bc7a09d55ba None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:17:41.654130 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d737306-42c8-49c5-8737-3bc7a09d55ba None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:17:41.662132 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:17:41.671996 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 296/605] 10.222.0.30 () {56 vars in 1148 bytes} [Tue Sep 6 04:17:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:17:41.671996 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d737306-42c8-49c5-8737-3bc7a09d55ba None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:41.673348 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f3fc38-a65d-44ba-b2d0-764b8e47ad3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:41.673993 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f3fc38-a65d-44ba-b2d0-764b8e47ad3d None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:41.674265 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f3fc38-a65d-44ba-b2d0-764b8e47ad3d None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:41.677872 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f3fc38-a65d-44ba-b2d0-764b8e47ad3d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:41.683870 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f3fc38-a65d-44ba-b2d0-764b8e47ad3d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:17:41.685366 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f3fc38-a65d-44ba-b2d0-764b8e47ad3d None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:17:41.699787 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-b8f3fc38-a65d-44ba-b2d0-764b8e47ad3d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 04:17:41.699787 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 04:17:41.699787 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 04:17:41.699787 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 04:17:41.699787 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 04:17:41.699787 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 04:17:41.699787 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 04:17:41.699787 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 04:17:41.699787 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 04:17:41.699787 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 04:17:41.699787 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 04:17:41.699787 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 04:17:41.699787 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 06 04:17:41.699787 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 06 04:17:41.699787 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 06 04:17:41.700733 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 06 04:17:41.700733 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 04:17:41.700733 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 04:17:41.700733 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 04:17:41.700733 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 04:17:41.700733 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 04:17:41.700733 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 04:17:41.700733 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 04:17:41.700733 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 04:17:41.700733 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 04:17:41.700733 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 04:17:41.700733 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 04:17:41.700733 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 04:17:41.700733 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 04:17:41.700733 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 04:17:41.700733 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 04:17:41.700733 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 04:17:41.702004 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 06 04:17:41.702004 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 06 04:17:41.702004 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 06 04:17:41.702004 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 06 04:17:41.702004 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 06 04:17:41.702004 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 06 04:17:41.702004 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 04:17:41.702004 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 04:17:41.709596 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 310/606] 10.222.0.30 () {56 vars in 1111 bytes} [Tue Sep 6 04:17:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 302 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 04:17:41.709596 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f3fc38-a65d-44ba-b2d0-764b8e47ad3d None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:41.737527 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:17:41.801038 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d737306-42c8-49c5-8737-3bc7a09d55ba None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:41.825139 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f3fc38-a65d-44ba-b2d0-764b8e47ad3d None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:41.838247 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d918dc68-a145-4c94-9fa5-444e88668502 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:41.838894 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d918dc68-a145-4c94-9fa5-444e88668502 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:41.839183 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d918dc68-a145-4c94-9fa5-444e88668502 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:41.839447 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d918dc68-a145-4c94-9fa5-444e88668502 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:41.850590 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d918dc68-a145-4c94-9fa5-444e88668502 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 04:17:41.855690 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-d918dc68-a145-4c94-9fa5-444e88668502 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 06 04:17:41.855690 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 04:17:41.855690 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 04:17:41.855690 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 04:17:41.855690 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 04:17:41.855690 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 04:17:41.855690 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 04:17:41.855690 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 04:17:41.855690 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 04:17:41.855690 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 04:17:41.855690 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 04:17:41.855690 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 04:17:41.855690 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 04:17:41.855690 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 04:17:41.855690 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 04:17:41.855690 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 04:17:41.856636 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 04:17:41.856636 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 04:17:41.856636 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 04:17:41.856636 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 04:17:41.856636 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 04:17:41.856636 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 04:17:41.856636 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 04:17:41.856636 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 04:17:41.856636 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 04:17:41.856636 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 04:17:41.856636 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 04:17:41.856636 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 04:17:41.856636 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 04:17:41.856636 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 04:17:41.856636 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 04:17:41.856636 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 04:17:41.857477 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 04:17:41.857477 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 04:17:41.857477 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 04:17:41.857477 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 04:17:41.857477 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 04:17:41.857477 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 04:17:41.857477 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 04:17:41.857477 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 04:17:41.857477 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 04:17:41.857477 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 04:17:41.857477 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 06 04:17:41.857477 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 04:17:41.861585 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 297/607] 10.222.0.30 () {56 vars in 1102 bytes} [Tue Sep 6 04:17:41 2022] GET /identity/v3/roles/service => generated 85 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 04:17:41.862554 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d918dc68-a145-4c94-9fa5-444e88668502 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:41.868861 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15e05eb-a678-4e61-88f6-35a2f172faeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:41.869840 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15e05eb-a678-4e61-88f6-35a2f172faeb None admin] REQUEST_METHOD: `PUT` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:41.870317 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15e05eb-a678-4e61-88f6-35a2f172faeb None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:41.870800 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15e05eb-a678-4e61-88f6-35a2f172faeb None admin] PATH_INFO: `/v3/projects/b707733ce65c42edbc1fd95ad0a1c06c/users/d09b5ab675634a2eb4b03755743f7d0d/roles/83cf0a194f8d45e792411e2d9afdfe50` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:41.888832 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15e05eb-a678-4e61-88f6-35a2f172faeb None admin] RBAC: Authorizing `identity:create_grant(project_id=b707733ce65c42edbc1fd95ad0a1c06c, user_id=d09b5ab675634a2eb4b03755743f7d0d, role_id=83cf0a194f8d45e792411e2d9afdfe50)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:17:41.891025 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15e05eb-a678-4e61-88f6-35a2f172faeb None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:17:41.970421 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:17:41.974008 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d918dc68-a145-4c94-9fa5-444e88668502 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:42.001572 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 311/608] 10.222.0.30 () {58 vars in 1439 bytes} [Tue Sep 6 04:17:41 2022] PUT /identity/v3/projects/b707733ce65c42edbc1fd95ad0a1c06c/users/d09b5ab675634a2eb4b03755743f7d0d/roles/83cf0a194f8d45e792411e2d9afdfe50 => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 04:17:42.003339 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15e05eb-a678-4e61-88f6-35a2f172faeb None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:42.035991 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e4beea-358e-4ebc-a5e0-d90990b4c5d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:42.037409 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e4beea-358e-4ebc-a5e0-d90990b4c5d5 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:42.039030 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e4beea-358e-4ebc-a5e0-d90990b4c5d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:42.041274 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e4beea-358e-4ebc-a5e0-d90990b4c5d5 None admin] PATH_INFO: `/v3/projects` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:42.042190 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e4beea-358e-4ebc-a5e0-d90990b4c5d5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:17:42.042602 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e4beea-358e-4ebc-a5e0-d90990b4c5d5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:17:42.044972 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e4beea-358e-4ebc-a5e0-d90990b4c5d5 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:17:42.075943 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 298/609] 10.222.0.30 () {56 vars in 1148 bytes} [Tue Sep 6 04:17:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:17:42.083913 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e4beea-358e-4ebc-a5e0-d90990b4c5d5 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:42.108330 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:17:42.177300 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15e05eb-a678-4e61-88f6-35a2f172faeb None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:42.221107 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa625d9a-e386-4eb5-81c1-fa84e02e16b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:42.221858 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa625d9a-e386-4eb5-81c1-fa84e02e16b4 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:42.222225 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa625d9a-e386-4eb5-81c1-fa84e02e16b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:42.223262 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa625d9a-e386-4eb5-81c1-fa84e02e16b4 None admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:42.223262 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa625d9a-e386-4eb5-81c1-fa84e02e16b4 None admin] RBAC: Adding query filter params (name=service) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:17:42.223829 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa625d9a-e386-4eb5-81c1-fa84e02e16b4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:17:42.225057 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa625d9a-e386-4eb5-81c1-fa84e02e16b4 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:17:42.233709 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e4beea-358e-4ebc-a5e0-d90990b4c5d5 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:42.240774 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 312/610] 10.222.0.30 () {56 vars in 1103 bytes} [Tue Sep 6 04:17:42 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:17:42.245275 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ec932f-3f42-4d80-9f23-0a3e6ca0396a None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:42.245275 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ec932f-3f42-4d80-9f23-0a3e6ca0396a None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:42.245275 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ec932f-3f42-4d80-9f23-0a3e6ca0396a None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:42.245275 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 313/611] 10.222.0.30 () {54 vars in 927 bytes} [Tue Sep 6 04:17:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 04:17:42.246285 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ec932f-3f42-4d80-9f23-0a3e6ca0396a None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:42.284008 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2feabb37-a249-40b7-b08b-9097947b40d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:42.286426 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2feabb37-a249-40b7-b08b-9097947b40d3 None admin] REQUEST_METHOD: `PUT` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:42.286869 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2feabb37-a249-40b7-b08b-9097947b40d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:42.287297 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2feabb37-a249-40b7-b08b-9097947b40d3 None admin] PATH_INFO: `/v3/projects/b707733ce65c42edbc1fd95ad0a1c06c/users/90492440f6a6471e8392361c098428aa/roles/83cf0a194f8d45e792411e2d9afdfe50` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:42.306846 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2feabb37-a249-40b7-b08b-9097947b40d3 None admin] RBAC: Authorizing `identity:create_grant(project_id=b707733ce65c42edbc1fd95ad0a1c06c, user_id=90492440f6a6471e8392361c098428aa, role_id=83cf0a194f8d45e792411e2d9afdfe50)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:17:42.309004 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2feabb37-a249-40b7-b08b-9097947b40d3 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:17:42.355750 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ec932f-3f42-4d80-9f23-0a3e6ca0396a None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:42.359016 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:17:42.382884 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 299/612] 10.222.0.30 () {58 vars in 1439 bytes} [Tue Sep 6 04:17:42 2022] PUT /identity/v3/projects/b707733ce65c42edbc1fd95ad0a1c06c/users/90492440f6a6471e8392361c098428aa/roles/83cf0a194f8d45e792411e2d9afdfe50 => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 04:17:42.384606 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c58dc53-5d92-48b9-b34c-da5333112695 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:42.385077 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c58dc53-5d92-48b9-b34c-da5333112695 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:42.385545 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c58dc53-5d92-48b9-b34c-da5333112695 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:42.411334 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-9c58dc53-5d92-48b9-b34c-da5333112695 None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:17:42.413034 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-9c58dc53-5d92-48b9-b34c-da5333112695 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:17:42.421611 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb620ae-7f99-48bb-8b27-005772a5a3e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:42.421611 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb620ae-7f99-48bb-8b27-005772a5a3e2 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:42.421611 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb620ae-7f99-48bb-8b27-005772a5a3e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:42.421611 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb620ae-7f99-48bb-8b27-005772a5a3e2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:42.422791 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:17:42.453530 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb620ae-7f99-48bb-8b27-005772a5a3e2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:17:42.453530 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb620ae-7f99-48bb-8b27-005772a5a3e2 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:17:42.496306 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-2fb620ae-7f99-48bb-8b27-005772a5a3e2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 04:17:42.496306 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 04:17:42.496306 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 04:17:42.496306 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 04:17:42.496306 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 04:17:42.496306 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 04:17:42.496306 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 04:17:42.496306 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 04:17:42.496306 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 04:17:42.496306 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 04:17:42.496306 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 04:17:42.496306 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 04:17:42.496306 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 04:17:42.496306 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 04:17:42.496306 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 04:17:42.496306 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 04:17:42.497534 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 04:17:42.497534 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 04:17:42.497534 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 04:17:42.497534 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 04:17:42.497534 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 04:17:42.497534 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 04:17:42.497534 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 04:17:42.497534 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 04:17:42.497534 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 04:17:42.497534 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 04:17:42.497534 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 04:17:42.497534 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 04:17:42.497534 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 04:17:42.497534 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 04:17:42.497534 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 04:17:42.497534 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 04:17:42.497534 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 04:17:42.498557 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 04:17:42.498557 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 04:17:42.498557 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 04:17:42.503238 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 314/613] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Sep 6 04:17:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 04:17:42.510957 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb620ae-7f99-48bb-8b27-005772a5a3e2 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:42.565560 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 300/614] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Sep 6 04:17:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 04:17:42.661048 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb620ae-7f99-48bb-8b27-005772a5a3e2 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:42.698999 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55ca77f-f3d4-427b-b9e5-3be096858855 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:42.699611 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55ca77f-f3d4-427b-b9e5-3be096858855 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:42.699882 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55ca77f-f3d4-427b-b9e5-3be096858855 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:42.700212 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55ca77f-f3d4-427b-b9e5-3be096858855 None admin] PATH_INFO: `/v3/domains` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:42.700861 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55ca77f-f3d4-427b-b9e5-3be096858855 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:17:42.701194 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55ca77f-f3d4-427b-b9e5-3be096858855 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:17:42.702627 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55ca77f-f3d4-427b-b9e5-3be096858855 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:17:42.724868 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 315/615] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Sep 6 04:17:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:17:42.730419 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c58dc53-5d92-48b9-b34c-da5333112695 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:42.849875 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbebfddc-ce16-4cb5-9b92-844074b6d888 None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:42.850385 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbebfddc-ce16-4cb5-9b92-844074b6d888 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:42.850385 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbebfddc-ce16-4cb5-9b92-844074b6d888 None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:42.851959 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 316/616] 10.222.0.30 () {54 vars in 927 bytes} [Tue Sep 6 04:17:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 04:17:42.857077 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be4ce1d-ecff-4ad1-8aca-70ca6395f0f6 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:42.857385 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be4ce1d-ecff-4ad1-8aca-70ca6395f0f6 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:42.857665 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be4ce1d-ecff-4ad1-8aca-70ca6395f0f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:42.878198 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-0be4ce1d-ecff-4ad1-8aca-70ca6395f0f6 None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:17:42.879433 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-0be4ce1d-ecff-4ad1-8aca-70ca6395f0f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:17:42.886384 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c58dc53-5d92-48b9-b34c-da5333112695 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:42.898679 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:17:42.936117 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc6c38b-3c3c-4019-903d-563b55896fe8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:42.937225 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc6c38b-3c3c-4019-903d-563b55896fe8 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:42.937681 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc6c38b-3c3c-4019-903d-563b55896fe8 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:42.938490 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc6c38b-3c3c-4019-903d-563b55896fe8 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:42.949238 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc6c38b-3c3c-4019-903d-563b55896fe8 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:17:42.951182 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc6c38b-3c3c-4019-903d-563b55896fe8 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:17:42.954987 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:17:42.959249 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 317/617] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Sep 6 04:17:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 06 04:17:42.965331 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be4ce1d-ecff-4ad1-8aca-70ca6395f0f6 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:42.980213 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-1bc6c38b-3c3c-4019-903d-563b55896fe8 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 06 04:17:42.980213 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 04:17:42.980213 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 04:17:42.980213 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 04:17:42.980213 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 04:17:42.980213 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 04:17:42.980213 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 04:17:42.980213 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 04:17:42.980213 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 04:17:42.980213 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 04:17:42.980213 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 04:17:42.980213 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 04:17:42.980213 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 06 04:17:42.980213 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 06 04:17:42.980213 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 06 04:17:42.980213 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 06 04:17:42.981173 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 04:17:42.981173 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 04:17:42.981173 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 06 04:17:42.981173 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 04:17:42.981173 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 06 04:17:42.981173 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 04:17:42.981173 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 04:17:42.981173 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 04:17:42.981173 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 04:17:42.981173 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 04:17:42.981173 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 04:17:42.981173 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 04:17:42.981173 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 04:17:42.981173 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 04:17:42.981173 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 04:17:42.981173 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 04:17:42.981173 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 04:17:42.981173 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 04:17:42.982147 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 04:17:42.982147 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 04:17:42.982147 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 06 04:17:42.982147 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 06 04:17:42.982147 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 06 04:17:42.982147 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 06 04:17:42.982147 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 06 04:17:42.982147 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 06 04:17:42.982147 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 06 04:17:42.982147 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 04:17:42.983486 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 301/618] 10.222.0.30 () {56 vars in 1096 bytes} [Tue Sep 6 04:17:42 2022] GET /identity/v3/users/swift => generated 83 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 04:17:42.990499 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc6c38b-3c3c-4019-903d-563b55896fe8 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:43.107947 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc6c38b-3c3c-4019-903d-563b55896fe8 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:43.147218 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5219f20-b83b-4fc1-80e9-168f7718626d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:43.148390 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5219f20-b83b-4fc1-80e9-168f7718626d None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:43.148881 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5219f20-b83b-4fc1-80e9-168f7718626d None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:43.149470 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5219f20-b83b-4fc1-80e9-168f7718626d None admin] PATH_INFO: `/v3/users` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:43.150470 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5219f20-b83b-4fc1-80e9-168f7718626d None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:17:43.150981 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5219f20-b83b-4fc1-80e9-168f7718626d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:17:43.153013 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5219f20-b83b-4fc1-80e9-168f7718626d None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:17:43.173732 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 302/619] 10.222.0.30 () {56 vars in 1135 bytes} [Tue Sep 6 04:17:42 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:17:43.180319 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5219f20-b83b-4fc1-80e9-168f7718626d None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:43.210505 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be4ce1d-ecff-4ad1-8aca-70ca6395f0f6 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:43.249277 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:17:43.296113 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5219f20-b83b-4fc1-80e9-168f7718626d None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:43.309347 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc52d51-d9e7-45e2-8dfb-f3b61d1fe139 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:43.310022 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc52d51-d9e7-45e2-8dfb-f3b61d1fe139 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:43.310312 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc52d51-d9e7-45e2-8dfb-f3b61d1fe139 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:43.310591 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc52d51-d9e7-45e2-8dfb-f3b61d1fe139 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:43.329967 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc52d51-d9e7-45e2-8dfb-f3b61d1fe139 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 04:17:43.336560 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-ccc52d51-d9e7-45e2-8dfb-f3b61d1fe139 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 06 04:17:43.336560 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 04:17:43.336560 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 04:17:43.336560 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 04:17:43.336560 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 04:17:43.336560 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 04:17:43.336560 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 04:17:43.336560 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 04:17:43.336560 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 04:17:43.336560 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 04:17:43.336560 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 04:17:43.336560 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 04:17:43.336560 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 04:17:43.336560 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 04:17:43.336560 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 04:17:43.336560 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 04:17:43.337759 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 04:17:43.337759 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 04:17:43.337759 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 04:17:43.337759 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 04:17:43.337759 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 04:17:43.337759 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 04:17:43.337759 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 04:17:43.337759 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 04:17:43.337759 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 04:17:43.337759 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 04:17:43.337759 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 04:17:43.337759 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 04:17:43.337759 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 04:17:43.337759 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 04:17:43.337759 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 04:17:43.337759 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 04:17:43.338601 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 04:17:43.338601 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 04:17:43.338601 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 04:17:43.338601 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 04:17:43.338601 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 04:17:43.338601 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 04:17:43.338601 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 04:17:43.338601 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 04:17:43.338601 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 04:17:43.338601 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 04:17:43.338601 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 06 04:17:43.338601 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 04:17:43.338601 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 318/620] 10.222.0.30 () {56 vars in 1102 bytes} [Tue Sep 6 04:17:42 2022] GET /identity/v3/roles/service => generated 85 bytes in 378 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 04:17:43.339331 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc52d51-d9e7-45e2-8dfb-f3b61d1fe139 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:43.340270 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2ab518-ee9e-40e1-8a01-1bc08b47e8f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:43.341193 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2ab518-ee9e-40e1-8a01-1bc08b47e8f9 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:43.341729 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2ab518-ee9e-40e1-8a01-1bc08b47e8f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:43.342379 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2ab518-ee9e-40e1-8a01-1bc08b47e8f9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:43.349133 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:17:43.355649 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b2ab518-ee9e-40e1-8a01-1bc08b47e8f9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:17:43.357581 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b2ab518-ee9e-40e1-8a01-1bc08b47e8f9 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:17:43.378070 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-0b2ab518-ee9e-40e1-8a01-1bc08b47e8f9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 04:17:43.378070 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 04:17:43.378070 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 04:17:43.378070 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 04:17:43.378070 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 04:17:43.378070 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 04:17:43.378070 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 04:17:43.378070 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 04:17:43.378070 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 04:17:43.378070 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 04:17:43.378070 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 04:17:43.378070 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 04:17:43.378070 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 04:17:43.378070 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 04:17:43.378070 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 04:17:43.378070 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 04:17:43.379854 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 04:17:43.379854 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 04:17:43.379854 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 04:17:43.379854 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 04:17:43.379854 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 04:17:43.379854 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 04:17:43.379854 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 04:17:43.379854 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 04:17:43.379854 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 04:17:43.379854 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 04:17:43.379854 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 04:17:43.379854 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 04:17:43.379854 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 04:17:43.379854 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 04:17:43.379854 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 04:17:43.379854 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 04:17:43.379854 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 04:17:43.380769 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 04:17:43.380769 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 04:17:43.380769 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 04:17:43.388961 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 303/621] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Sep 6 04:17:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 04:17:43.390055 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2ab518-ee9e-40e1-8a01-1bc08b47e8f9 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:43.463845 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc52d51-d9e7-45e2-8dfb-f3b61d1fe139 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:43.500633 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67cfdf6d-39a1-4207-9a90-d0c9ec85b211 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:43.501525 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cfdf6d-39a1-4207-9a90-d0c9ec85b211 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:43.502216 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cfdf6d-39a1-4207-9a90-d0c9ec85b211 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:43.502662 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cfdf6d-39a1-4207-9a90-d0c9ec85b211 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:43.513569 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67cfdf6d-39a1-4207-9a90-d0c9ec85b211 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 04:17:43.518767 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-67cfdf6d-39a1-4207-9a90-d0c9ec85b211 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 06 04:17:43.518767 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 04:17:43.518767 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 04:17:43.518767 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 04:17:43.518767 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 04:17:43.518767 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 04:17:43.518767 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 04:17:43.518767 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 04:17:43.518767 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 04:17:43.518767 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 04:17:43.518767 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 04:17:43.518767 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 04:17:43.518767 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 04:17:43.518767 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 04:17:43.518767 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 04:17:43.518767 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 04:17:43.520236 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 04:17:43.520236 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 04:17:43.520236 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 04:17:43.520236 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 04:17:43.520236 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 04:17:43.520236 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 04:17:43.520236 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 04:17:43.520236 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 04:17:43.520236 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 04:17:43.520236 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 04:17:43.520236 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 04:17:43.520236 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 04:17:43.520236 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 04:17:43.520236 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 04:17:43.520236 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 04:17:43.520236 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 04:17:43.521095 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 04:17:43.521095 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 04:17:43.521095 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 04:17:43.521095 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 04:17:43.521095 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 04:17:43.521095 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 04:17:43.521095 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 04:17:43.521095 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 04:17:43.521095 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 04:17:43.521095 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 04:17:43.521095 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 06 04:17:43.521095 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 04:17:43.522239 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2ab518-ee9e-40e1-8a01-1bc08b47e8f9 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:43.523864 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 319/622] 10.222.0.30 () {56 vars in 1102 bytes} [Tue Sep 6 04:17:43 2022] GET /identity/v3/roles/service => generated 85 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 04:17:43.524936 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67cfdf6d-39a1-4207-9a90-d0c9ec85b211 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:43.549732 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:17:43.565092 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5649efc6-eaa2-4293-9bf7-c07a85094add None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:43.568828 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5649efc6-eaa2-4293-9bf7-c07a85094add None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:43.568828 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5649efc6-eaa2-4293-9bf7-c07a85094add None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:43.568828 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5649efc6-eaa2-4293-9bf7-c07a85094add None admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:43.568828 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5649efc6-eaa2-4293-9bf7-c07a85094add None admin] RBAC: Adding query filter params (name=service) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:17:43.568828 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5649efc6-eaa2-4293-9bf7-c07a85094add None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:17:43.570006 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5649efc6-eaa2-4293-9bf7-c07a85094add None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:17:43.579789 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 304/623] 10.222.0.30 () {56 vars in 1103 bytes} [Tue Sep 6 04:17:43 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:17:43.580811 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5649efc6-eaa2-4293-9bf7-c07a85094add None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:43.601282 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:17:43.639326 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67cfdf6d-39a1-4207-9a90-d0c9ec85b211 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:43.676044 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301b95af-6aad-4cfa-8a86-cb349bf2b89d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:43.677117 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301b95af-6aad-4cfa-8a86-cb349bf2b89d None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:43.677605 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301b95af-6aad-4cfa-8a86-cb349bf2b89d None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:43.678093 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301b95af-6aad-4cfa-8a86-cb349bf2b89d None admin] PATH_INFO: `/v3/domains` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:43.678921 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301b95af-6aad-4cfa-8a86-cb349bf2b89d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:17:43.679406 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301b95af-6aad-4cfa-8a86-cb349bf2b89d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:17:43.680971 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301b95af-6aad-4cfa-8a86-cb349bf2b89d None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:17:43.691883 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5649efc6-eaa2-4293-9bf7-c07a85094add None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:43.698027 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 320/624] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Sep 6 04:17:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:17:43.699151 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301b95af-6aad-4cfa-8a86-cb349bf2b89d None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:43.736891 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09aba07-ebeb-4b3d-86d7-afcc20f9218a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:43.737787 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09aba07-ebeb-4b3d-86d7-afcc20f9218a None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:43.738027 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09aba07-ebeb-4b3d-86d7-afcc20f9218a None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:43.738502 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09aba07-ebeb-4b3d-86d7-afcc20f9218a None admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:43.739268 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09aba07-ebeb-4b3d-86d7-afcc20f9218a None admin] RBAC: Adding query filter params (name=service) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:17:43.739830 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09aba07-ebeb-4b3d-86d7-afcc20f9218a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:17:43.741499 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09aba07-ebeb-4b3d-86d7-afcc20f9218a None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:17:43.755862 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 305/625] 10.222.0.30 () {56 vars in 1103 bytes} [Tue Sep 6 04:17:43 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:17:43.756729 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09aba07-ebeb-4b3d-86d7-afcc20f9218a None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:43.809754 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:17:43.828307 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301b95af-6aad-4cfa-8a86-cb349bf2b89d None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:43.855661 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:17:43.865662 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9571e5-45b0-483f-becc-b4499260f529 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:43.865906 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09aba07-ebeb-4b3d-86d7-afcc20f9218a None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:43.866706 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9571e5-45b0-483f-becc-b4499260f529 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:43.867132 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9571e5-45b0-483f-becc-b4499260f529 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:43.867555 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9571e5-45b0-483f-becc-b4499260f529 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:43.879243 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b9571e5-45b0-483f-becc-b4499260f529 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:17:43.881008 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b9571e5-45b0-483f-becc-b4499260f529 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:17:43.895445 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-7b9571e5-45b0-483f-becc-b4499260f529 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 04:17:43.895445 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 04:17:43.895445 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 04:17:43.895445 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 04:17:43.895445 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 04:17:43.895445 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 04:17:43.895445 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 04:17:43.895445 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 04:17:43.895445 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 04:17:43.895445 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 04:17:43.895445 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 04:17:43.895445 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 04:17:43.895445 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 04:17:43.895445 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 04:17:43.895445 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 04:17:43.895445 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 04:17:43.896416 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 04:17:43.896416 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 04:17:43.896416 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 04:17:43.896416 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 04:17:43.896416 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 04:17:43.896416 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 04:17:43.896416 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 04:17:43.896416 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 04:17:43.896416 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 04:17:43.896416 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 04:17:43.896416 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 04:17:43.896416 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 04:17:43.896416 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 04:17:43.896416 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 04:17:43.896416 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 04:17:43.896416 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 04:17:43.896416 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 04:17:43.897398 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 04:17:43.897398 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 04:17:43.897398 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 04:17:43.899752 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 321/626] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Sep 6 04:17:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 04:17:43.900884 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9571e5-45b0-483f-becc-b4499260f529 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:43.910821 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879733c7-d52a-4113-bd6f-e3e7c38c70ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:43.911507 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879733c7-d52a-4113-bd6f-e3e7c38c70ab None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:43.911856 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879733c7-d52a-4113-bd6f-e3e7c38c70ab None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:43.912138 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879733c7-d52a-4113-bd6f-e3e7c38c70ab None admin] PATH_INFO: `/v3/projects/service` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:43.919856 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-879733c7-d52a-4113-bd6f-e3e7c38c70ab None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:17:43.921447 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-879733c7-d52a-4113-bd6f-e3e7c38c70ab None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:17:43.934747 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-879733c7-d52a-4113-bd6f-e3e7c38c70ab None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 04:17:43.934747 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 04:17:43.934747 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 04:17:43.934747 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 04:17:43.934747 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 04:17:43.934747 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 04:17:43.934747 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 04:17:43.934747 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 04:17:43.934747 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 04:17:43.934747 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 04:17:43.934747 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 04:17:43.934747 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 04:17:43.934747 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 06 04:17:43.934747 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 06 04:17:43.934747 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 06 04:17:43.935676 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 06 04:17:43.935676 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 04:17:43.935676 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 04:17:43.935676 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 04:17:43.935676 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 04:17:43.935676 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 04:17:43.935676 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 04:17:43.935676 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 04:17:43.935676 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 04:17:43.935676 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 04:17:43.935676 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 04:17:43.935676 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 04:17:43.935676 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 04:17:43.935676 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 04:17:43.935676 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 04:17:43.935676 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 04:17:43.935676 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 04:17:43.936574 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 06 04:17:43.936574 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 06 04:17:43.936574 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 06 04:17:43.936574 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 06 04:17:43.936574 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 06 04:17:43.936574 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 06 04:17:43.936574 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 04:17:43.936574 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 04:17:43.936574 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 306/627] 10.222.0.30 () {56 vars in 1111 bytes} [Tue Sep 6 04:17:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 04:17:43.941041 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879733c7-d52a-4113-bd6f-e3e7c38c70ab None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:44.016874 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9571e5-45b0-483f-becc-b4499260f529 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:44.044151 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:17:44.051888 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879733c7-d52a-4113-bd6f-e3e7c38c70ab None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:44.057856 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480c17d2-520f-4417-8acc-58ae00e8b3c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:44.058773 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480c17d2-520f-4417-8acc-58ae00e8b3c1 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:44.059257 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480c17d2-520f-4417-8acc-58ae00e8b3c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:44.059729 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480c17d2-520f-4417-8acc-58ae00e8b3c1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:44.071282 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480c17d2-520f-4417-8acc-58ae00e8b3c1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:17:44.073013 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480c17d2-520f-4417-8acc-58ae00e8b3c1 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:17:44.088202 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de097f36-0437-4bc6-818e-adabe38c578e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:44.088828 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de097f36-0437-4bc6-818e-adabe38c578e None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:44.089112 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de097f36-0437-4bc6-818e-adabe38c578e None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:44.089473 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de097f36-0437-4bc6-818e-adabe38c578e None admin] PATH_INFO: `/v3/domains` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:44.090127 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de097f36-0437-4bc6-818e-adabe38c578e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:17:44.090432 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de097f36-0437-4bc6-818e-adabe38c578e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:17:44.090857 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-480c17d2-520f-4417-8acc-58ae00e8b3c1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 04:17:44.090857 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 04:17:44.090857 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 04:17:44.090857 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 04:17:44.090857 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 04:17:44.090857 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 04:17:44.090857 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 04:17:44.090857 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 04:17:44.090857 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 04:17:44.090857 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 04:17:44.090857 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 04:17:44.090857 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 04:17:44.090857 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 04:17:44.090857 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 04:17:44.090857 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 04:17:44.090857 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 04:17:44.091760 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 04:17:44.091760 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 04:17:44.091760 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 04:17:44.091760 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 04:17:44.091760 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 04:17:44.091760 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 04:17:44.091760 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 04:17:44.091760 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 04:17:44.091760 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 04:17:44.091760 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 04:17:44.091760 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 04:17:44.091760 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 04:17:44.091760 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 04:17:44.091760 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 04:17:44.091760 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 04:17:44.091760 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 04:17:44.091760 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 04:17:44.092645 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 04:17:44.092645 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 04:17:44.092645 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 04:17:44.092823 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de097f36-0437-4bc6-818e-adabe38c578e None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:17:44.094511 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 322/628] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Sep 6 04:17:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 04:17:44.095670 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480c17d2-520f-4417-8acc-58ae00e8b3c1 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:44.109663 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 307/629] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Sep 6 04:17:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:17:44.110683 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de097f36-0437-4bc6-818e-adabe38c578e None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:44.134600 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:17:44.210923 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480c17d2-520f-4417-8acc-58ae00e8b3c1 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:44.221718 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de097f36-0437-4bc6-818e-adabe38c578e None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:44.246819 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b583e58-42af-45b4-8256-8a3467f5f7cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:44.247552 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b583e58-42af-45b4-8256-8a3467f5f7cc None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:44.247835 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b583e58-42af-45b4-8256-8a3467f5f7cc None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:44.248115 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b583e58-42af-45b4-8256-8a3467f5f7cc None admin] PATH_INFO: `/v3/projects` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:44.248812 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b583e58-42af-45b4-8256-8a3467f5f7cc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:17:44.249118 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b583e58-42af-45b4-8256-8a3467f5f7cc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:17:44.250591 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b583e58-42af-45b4-8256-8a3467f5f7cc None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:17:44.260523 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28348201-713e-4b36-9ee0-df0033755025 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:44.261158 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28348201-713e-4b36-9ee0-df0033755025 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:44.261615 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28348201-713e-4b36-9ee0-df0033755025 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:44.261895 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28348201-713e-4b36-9ee0-df0033755025 None admin] PATH_INFO: `/v3/domains` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:44.262548 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28348201-713e-4b36-9ee0-df0033755025 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:17:44.262853 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28348201-713e-4b36-9ee0-df0033755025 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:17:44.263508 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 323/630] 10.222.0.30 () {56 vars in 1148 bytes} [Tue Sep 6 04:17:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:17:44.264269 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28348201-713e-4b36-9ee0-df0033755025 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:17:44.268869 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b583e58-42af-45b4-8256-8a3467f5f7cc None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:44.280732 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 308/631] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Sep 6 04:17:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:17:44.280732 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b857e8-7f7f-4540-8a66-5f4457a8e098 None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:44.280732 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b857e8-7f7f-4540-8a66-5f4457a8e098 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:44.280732 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b857e8-7f7f-4540-8a66-5f4457a8e098 None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:44.280732 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 309/632] 10.222.0.30 () {54 vars in 927 bytes} [Tue Sep 6 04:17:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 04:17:44.280732 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b857e8-7f7f-4540-8a66-5f4457a8e098 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:44.296758 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:17:44.373433 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:17:44.388894 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b583e58-42af-45b4-8256-8a3467f5f7cc None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:44.393828 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b857e8-7f7f-4540-8a66-5f4457a8e098 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:44.427661 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8cbe9fc-30e5-42b1-916e-e6f610d9dac6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:44.428329 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cbe9fc-30e5-42b1-916e-e6f610d9dac6 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:44.428563 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cbe9fc-30e5-42b1-916e-e6f610d9dac6 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:44.428834 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cbe9fc-30e5-42b1-916e-e6f610d9dac6 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:44.435136 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bccffb7b-9e06-449e-adac-98e4ad30c02a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:44.435810 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccffb7b-9e06-449e-adac-98e4ad30c02a None admin] REQUEST_METHOD: `PUT` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:44.436100 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccffb7b-9e06-449e-adac-98e4ad30c02a None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:44.436376 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccffb7b-9e06-449e-adac-98e4ad30c02a None admin] PATH_INFO: `/v3/projects/b707733ce65c42edbc1fd95ad0a1c06c/users/a676ec7f92a144edbeca8e6029ba9ee6/roles/83cf0a194f8d45e792411e2d9afdfe50` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:44.436913 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8cbe9fc-30e5-42b1-916e-e6f610d9dac6 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:17:44.438393 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8cbe9fc-30e5-42b1-916e-e6f610d9dac6 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:17:44.451672 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-b8cbe9fc-30e5-42b1-916e-e6f610d9dac6 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 06 04:17:44.451672 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 04:17:44.451672 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 04:17:44.451672 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 04:17:44.451672 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 04:17:44.451672 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 04:17:44.451672 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 04:17:44.451672 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 04:17:44.451672 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 04:17:44.451672 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 04:17:44.451672 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 04:17:44.451672 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 04:17:44.451672 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 06 04:17:44.451672 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 06 04:17:44.451672 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 06 04:17:44.451672 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 06 04:17:44.452659 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 04:17:44.452659 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 04:17:44.452659 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 06 04:17:44.452659 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 04:17:44.452659 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 06 04:17:44.452659 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 04:17:44.452659 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 04:17:44.452659 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 04:17:44.452659 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 04:17:44.452659 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 04:17:44.452659 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 04:17:44.452659 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 04:17:44.452659 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 04:17:44.452659 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 04:17:44.452659 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 04:17:44.452659 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 04:17:44.452659 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 04:17:44.452659 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 04:17:44.453689 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 04:17:44.453689 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 04:17:44.453689 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 06 04:17:44.453689 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 06 04:17:44.453689 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 06 04:17:44.453689 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 06 04:17:44.453689 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 06 04:17:44.453689 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 06 04:17:44.453689 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 06 04:17:44.453689 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 04:17:44.453689 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 324/633] 10.222.0.30 () {56 vars in 1093 bytes} [Tue Sep 6 04:17:44 2022] GET /identity/v3/users/nova => generated 82 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 04:17:44.453689 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8cbe9fc-30e5-42b1-916e-e6f610d9dac6 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:44.458287 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bccffb7b-9e06-449e-adac-98e4ad30c02a None admin] RBAC: Authorizing `identity:create_grant(project_id=b707733ce65c42edbc1fd95ad0a1c06c, user_id=a676ec7f92a144edbeca8e6029ba9ee6, role_id=83cf0a194f8d45e792411e2d9afdfe50)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:17:44.459766 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bccffb7b-9e06-449e-adac-98e4ad30c02a None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:17:44.554533 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8cbe9fc-30e5-42b1-916e-e6f610d9dac6 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:44.567866 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 310/634] 10.222.0.30 () {58 vars in 1439 bytes} [Tue Sep 6 04:17:44 2022] PUT /identity/v3/projects/b707733ce65c42edbc1fd95ad0a1c06c/users/a676ec7f92a144edbeca8e6029ba9ee6/roles/83cf0a194f8d45e792411e2d9afdfe50 => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 04:17:44.579081 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef125b5-7aee-4b0a-97fe-0766086c85b7 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:44.579732 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef125b5-7aee-4b0a-97fe-0766086c85b7 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:44.579732 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef125b5-7aee-4b0a-97fe-0766086c85b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:44.592663 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f328113-5021-4aaf-8a61-4f3c5321abad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:44.593328 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f328113-5021-4aaf-8a61-4f3c5321abad None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:44.593614 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f328113-5021-4aaf-8a61-4f3c5321abad None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:44.593889 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f328113-5021-4aaf-8a61-4f3c5321abad None admin] PATH_INFO: `/v3/users/glance` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:44.603943 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f328113-5021-4aaf-8a61-4f3c5321abad None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:17:44.605541 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f328113-5021-4aaf-8a61-4f3c5321abad None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:17:44.607180 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-fef125b5-7aee-4b0a-97fe-0766086c85b7 None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:17:44.608509 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-fef125b5-7aee-4b0a-97fe-0766086c85b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:17:44.619355 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-8f328113-5021-4aaf-8a61-4f3c5321abad None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 06 04:17:44.619355 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 04:17:44.619355 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 04:17:44.619355 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 04:17:44.619355 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 04:17:44.619355 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 04:17:44.619355 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 04:17:44.619355 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 04:17:44.619355 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 04:17:44.619355 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 04:17:44.619355 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 04:17:44.619355 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 04:17:44.619355 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 06 04:17:44.619355 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 06 04:17:44.619355 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 06 04:17:44.619355 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 06 04:17:44.620616 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 04:17:44.620616 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 04:17:44.620616 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 06 04:17:44.620616 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 04:17:44.620616 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 06 04:17:44.620616 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 04:17:44.620616 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 04:17:44.620616 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 04:17:44.620616 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 04:17:44.620616 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 04:17:44.620616 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 04:17:44.620616 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 04:17:44.620616 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 04:17:44.620616 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 04:17:44.620616 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 04:17:44.620616 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 04:17:44.620616 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 04:17:44.620616 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 04:17:44.621739 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 04:17:44.621739 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 04:17:44.621739 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 06 04:17:44.621739 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 06 04:17:44.621739 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 06 04:17:44.621739 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 06 04:17:44.621739 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 06 04:17:44.621739 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 06 04:17:44.621739 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 06 04:17:44.621739 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 04:17:44.621739 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 325/635] 10.222.0.30 () {56 vars in 1099 bytes} [Tue Sep 6 04:17:44 2022] GET /identity/v3/users/glance => generated 84 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 04:17:44.626545 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f328113-5021-4aaf-8a61-4f3c5321abad None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:44.646045 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:17:44.688084 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:17:44.704834 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 311/636] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Sep 6 04:17:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 04:17:44.704834 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5001dae4-a6d9-423b-93b4-f06a75d3a5a8 None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:44.704834 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5001dae4-a6d9-423b-93b4-f06a75d3a5a8 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:44.704834 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5001dae4-a6d9-423b-93b4-f06a75d3a5a8 None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:44.704834 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 312/637] 10.222.0.30 () {54 vars in 927 bytes} [Tue Sep 6 04:17:44 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 04:17:44.708668 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5001dae4-a6d9-423b-93b4-f06a75d3a5a8 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:44.750143 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f328113-5021-4aaf-8a61-4f3c5321abad None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:44.798809 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c8e8dd-0a04-43ce-bf43-0983adfebef7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:44.799928 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c8e8dd-0a04-43ce-bf43-0983adfebef7 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:44.800488 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c8e8dd-0a04-43ce-bf43-0983adfebef7 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:44.800918 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c8e8dd-0a04-43ce-bf43-0983adfebef7 None admin] PATH_INFO: `/v3/users` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:44.802148 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c8e8dd-0a04-43ce-bf43-0983adfebef7 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:17:44.803474 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c8e8dd-0a04-43ce-bf43-0983adfebef7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:17:44.805112 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c8e8dd-0a04-43ce-bf43-0983adfebef7 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:17:44.828944 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 326/638] 10.222.0.30 () {56 vars in 1133 bytes} [Tue Sep 6 04:17:44 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:17:44.831194 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8706642c-3737-4739-ba56-2dcbc4480429 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:44.832161 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8706642c-3737-4739-ba56-2dcbc4480429 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:44.832599 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8706642c-3737-4739-ba56-2dcbc4480429 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:44.838219 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5001dae4-a6d9-423b-93b4-f06a75d3a5a8 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:44.854979 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-8706642c-3737-4739-ba56-2dcbc4480429 None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:17:44.857117 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-8706642c-3737-4739-ba56-2dcbc4480429 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:17:44.879853 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36ab5a0-46a9-4b35-a5a6-0317d0dd7fe2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:44.880479 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36ab5a0-46a9-4b35-a5a6-0317d0dd7fe2 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:44.880760 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36ab5a0-46a9-4b35-a5a6-0317d0dd7fe2 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:44.881087 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36ab5a0-46a9-4b35-a5a6-0317d0dd7fe2 None admin] PATH_INFO: `/v3/users` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:44.882013 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f36ab5a0-46a9-4b35-a5a6-0317d0dd7fe2 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:17:44.882321 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f36ab5a0-46a9-4b35-a5a6-0317d0dd7fe2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:17:44.883869 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f36ab5a0-46a9-4b35-a5a6-0317d0dd7fe2 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:17:44.907667 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 313/639] 10.222.0.30 () {56 vars in 1137 bytes} [Tue Sep 6 04:17:44 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:17:44.907667 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36ab5a0-46a9-4b35-a5a6-0317d0dd7fe2 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:44.942989 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:17:44.955675 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 327/640] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Sep 6 04:17:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 04:17:44.957266 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8706642c-3737-4739-ba56-2dcbc4480429 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:44.976523 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:17:45.029808 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36ab5a0-46a9-4b35-a5a6-0317d0dd7fe2 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:45.068703 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8706642c-3737-4739-ba56-2dcbc4480429 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:45.069578 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d014921-0dab-4b22-8206-8cad0766dc8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:45.070404 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d014921-0dab-4b22-8206-8cad0766dc8f None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:45.070786 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d014921-0dab-4b22-8206-8cad0766dc8f None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:45.071228 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d014921-0dab-4b22-8206-8cad0766dc8f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:45.106130 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ec5ca0-5965-459b-adc1-c4de411b2570 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:45.106756 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ec5ca0-5965-459b-adc1-c4de411b2570 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:45.107037 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ec5ca0-5965-459b-adc1-c4de411b2570 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:45.107313 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ec5ca0-5965-459b-adc1-c4de411b2570 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:45.150811 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d014921-0dab-4b22-8206-8cad0766dc8f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:17:45.152530 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d014921-0dab-4b22-8206-8cad0766dc8f None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:17:45.177971 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:17:45.181659 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ec5ca0-5965-459b-adc1-c4de411b2570 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:17:45.183380 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ec5ca0-5965-459b-adc1-c4de411b2570 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:17:45.213686 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:17:45.268490 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 314/641] 10.222.0.30 () {58 vars in 1282 bytes} [Tue Sep 6 04:17:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 06 04:17:45.269572 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d014921-0dab-4b22-8206-8cad0766dc8f None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:45.309724 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 328/642] 10.222.0.30 () {58 vars in 1282 bytes} [Tue Sep 6 04:17:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 06 04:17:45.310569 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ec5ca0-5965-459b-adc1-c4de411b2570 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:45.377759 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d014921-0dab-4b22-8206-8cad0766dc8f None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:45.416004 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0243bd12-b02a-42e6-9414-88d7e0723730 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:45.431708 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0243bd12-b02a-42e6-9414-88d7e0723730 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:45.431708 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0243bd12-b02a-42e6-9414-88d7e0723730 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:45.431708 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0243bd12-b02a-42e6-9414-88d7e0723730 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:45.432009 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:17:45.432009 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ec5ca0-5965-459b-adc1-c4de411b2570 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:45.432195 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0243bd12-b02a-42e6-9414-88d7e0723730 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 04:17:45.433755 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-0243bd12-b02a-42e6-9414-88d7e0723730 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 06 04:17:45.433755 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 04:17:45.433755 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 04:17:45.433755 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 04:17:45.433755 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 04:17:45.433755 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 04:17:45.433755 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 04:17:45.433755 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 04:17:45.433755 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 04:17:45.433755 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 04:17:45.433755 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 04:17:45.433755 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 04:17:45.433755 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 04:17:45.433755 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 04:17:45.433755 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 04:17:45.433755 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 04:17:45.434832 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 04:17:45.434832 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 04:17:45.434832 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 04:17:45.434832 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 04:17:45.434832 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 04:17:45.434832 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 04:17:45.434832 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 04:17:45.434832 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 04:17:45.434832 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 04:17:45.434832 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 04:17:45.434832 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 04:17:45.434832 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 04:17:45.434832 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 04:17:45.434832 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 04:17:45.434832 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 04:17:45.434832 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 04:17:45.441251 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 04:17:45.441251 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 04:17:45.441251 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 04:17:45.441251 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 04:17:45.441251 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 04:17:45.441251 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 04:17:45.441251 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 04:17:45.441251 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 04:17:45.441251 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 04:17:45.441251 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 04:17:45.441251 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 06 04:17:45.441251 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 04:17:45.441251 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 315/643] 10.222.0.30 () {56 vars in 1102 bytes} [Tue Sep 6 04:17:45 2022] GET /identity/v3/roles/service => generated 85 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 04:17:45.441251 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0243bd12-b02a-42e6-9414-88d7e0723730 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:45.453832 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:17:45.466976 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844a6c61-ab3f-452c-a8fa-9e451b5389d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:45.467795 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844a6c61-ab3f-452c-a8fa-9e451b5389d6 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:45.468071 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844a6c61-ab3f-452c-a8fa-9e451b5389d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:45.468351 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844a6c61-ab3f-452c-a8fa-9e451b5389d6 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:45.480897 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844a6c61-ab3f-452c-a8fa-9e451b5389d6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 04:17:45.486624 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-844a6c61-ab3f-452c-a8fa-9e451b5389d6 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 06 04:17:45.486624 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 04:17:45.486624 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 04:17:45.486624 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 04:17:45.486624 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 04:17:45.486624 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 04:17:45.486624 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 04:17:45.486624 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 04:17:45.486624 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 04:17:45.486624 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 04:17:45.486624 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 04:17:45.486624 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 04:17:45.486624 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 04:17:45.486624 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 04:17:45.486624 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 04:17:45.486624 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 04:17:45.487952 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 04:17:45.487952 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 04:17:45.487952 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 04:17:45.487952 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 04:17:45.487952 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 04:17:45.487952 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 04:17:45.487952 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 04:17:45.487952 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 04:17:45.487952 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 04:17:45.487952 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 04:17:45.487952 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 04:17:45.487952 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 04:17:45.487952 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 04:17:45.487952 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 04:17:45.487952 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 04:17:45.487952 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 04:17:45.489008 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 04:17:45.489008 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 04:17:45.489008 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 04:17:45.489008 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 04:17:45.489008 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 04:17:45.489008 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 04:17:45.489008 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 04:17:45.489008 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 04:17:45.489008 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 04:17:45.489008 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 04:17:45.489008 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 06 04:17:45.489008 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 04:17:45.490194 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 329/644] 10.222.0.30 () {56 vars in 1102 bytes} [Tue Sep 6 04:17:45 2022] GET /identity/v3/roles/service => generated 85 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 04:17:45.491090 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844a6c61-ab3f-452c-a8fa-9e451b5389d6 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:45.551147 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0243bd12-b02a-42e6-9414-88d7e0723730 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:45.587721 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057e0b54-cbbe-44e8-9946-bb0382288bc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:45.589069 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057e0b54-cbbe-44e8-9946-bb0382288bc6 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:45.589629 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057e0b54-cbbe-44e8-9946-bb0382288bc6 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:45.590036 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057e0b54-cbbe-44e8-9946-bb0382288bc6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:45.593134 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844a6c61-ab3f-452c-a8fa-9e451b5389d6 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:45.601524 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057e0b54-cbbe-44e8-9946-bb0382288bc6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:17:45.602984 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057e0b54-cbbe-44e8-9946-bb0382288bc6 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:17:45.618414 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-057e0b54-cbbe-44e8-9946-bb0382288bc6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 04:17:45.618414 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 04:17:45.618414 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 04:17:45.618414 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 04:17:45.618414 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 04:17:45.618414 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 04:17:45.618414 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 04:17:45.618414 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 04:17:45.618414 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 04:17:45.618414 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 04:17:45.618414 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 04:17:45.618414 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 04:17:45.618414 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 04:17:45.618414 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 04:17:45.618414 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 04:17:45.618414 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 04:17:45.619414 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 04:17:45.619414 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 04:17:45.619414 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 04:17:45.619414 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 04:17:45.619414 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 04:17:45.619414 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 04:17:45.619414 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 04:17:45.619414 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 04:17:45.619414 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 04:17:45.619414 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 04:17:45.619414 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 04:17:45.619414 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 04:17:45.619414 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 04:17:45.619414 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 04:17:45.619414 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 04:17:45.619414 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 04:17:45.619414 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 04:17:45.620477 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 04:17:45.620477 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 04:17:45.620477 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 04:17:45.620477 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 316/645] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Sep 6 04:17:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 04:17:45.620477 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057e0b54-cbbe-44e8-9946-bb0382288bc6 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:45.645017 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17305ac1-d193-4539-b465-1ac891c615a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:45.645710 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17305ac1-d193-4539-b465-1ac891c615a8 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:45.646002 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17305ac1-d193-4539-b465-1ac891c615a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:45.646185 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17305ac1-d193-4539-b465-1ac891c615a8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:45.658405 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17305ac1-d193-4539-b465-1ac891c615a8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:17:45.660351 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17305ac1-d193-4539-b465-1ac891c615a8 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:17:45.675069 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-17305ac1-d193-4539-b465-1ac891c615a8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 04:17:45.675069 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 04:17:45.675069 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 04:17:45.675069 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 04:17:45.675069 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 04:17:45.675069 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 04:17:45.675069 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 04:17:45.675069 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 04:17:45.675069 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 04:17:45.675069 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 04:17:45.675069 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 04:17:45.675069 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 04:17:45.675069 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 04:17:45.675069 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 04:17:45.675069 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 04:17:45.675069 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 04:17:45.681659 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 04:17:45.681659 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 04:17:45.681659 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 04:17:45.681659 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 04:17:45.681659 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 04:17:45.681659 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 04:17:45.681659 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 04:17:45.681659 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 04:17:45.681659 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 04:17:45.681659 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 04:17:45.681659 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 04:17:45.681659 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 04:17:45.681659 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 04:17:45.681659 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 04:17:45.681659 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 04:17:45.681659 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 04:17:45.681659 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 04:17:45.682556 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 04:17:45.682556 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 04:17:45.682556 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 04:17:45.682556 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 330/646] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Sep 6 04:17:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 04:17:45.682556 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17305ac1-d193-4539-b465-1ac891c615a8 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:45.691877 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:17:45.694434 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:17:45.740499 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057e0b54-cbbe-44e8-9946-bb0382288bc6 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:45.775306 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c624c7a7-0211-4801-838d-e2ce18d769d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:45.775918 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c624c7a7-0211-4801-838d-e2ce18d769d0 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:45.776196 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c624c7a7-0211-4801-838d-e2ce18d769d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:45.776469 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c624c7a7-0211-4801-838d-e2ce18d769d0 None admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:45.777123 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c624c7a7-0211-4801-838d-e2ce18d769d0 None admin] RBAC: Adding query filter params (name=service) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:17:45.777527 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c624c7a7-0211-4801-838d-e2ce18d769d0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:17:45.778890 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c624c7a7-0211-4801-838d-e2ce18d769d0 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:17:45.787258 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 317/647] 10.222.0.30 () {56 vars in 1103 bytes} [Tue Sep 6 04:17:45 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:17:45.788050 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c624c7a7-0211-4801-838d-e2ce18d769d0 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:45.799175 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17305ac1-d193-4539-b465-1ac891c615a8 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:45.837164 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f205ebf2-5bb4-4718-994d-03af0f50c098 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:45.837883 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f205ebf2-5bb4-4718-994d-03af0f50c098 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:45.838236 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f205ebf2-5bb4-4718-994d-03af0f50c098 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:45.838582 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f205ebf2-5bb4-4718-994d-03af0f50c098 None admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:45.839333 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f205ebf2-5bb4-4718-994d-03af0f50c098 None admin] RBAC: Adding query filter params (name=service) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:17:45.839702 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f205ebf2-5bb4-4718-994d-03af0f50c098 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:17:45.841171 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f205ebf2-5bb4-4718-994d-03af0f50c098 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:17:45.853874 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 331/648] 10.222.0.30 () {56 vars in 1103 bytes} [Tue Sep 6 04:17:45 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:17:45.853874 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f205ebf2-5bb4-4718-994d-03af0f50c098 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:45.890660 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c624c7a7-0211-4801-838d-e2ce18d769d0 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:45.916125 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:17:45.927120 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000405d3-4296-4345-89aa-b5211edba587 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:45.927739 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000405d3-4296-4345-89aa-b5211edba587 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:45.928507 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000405d3-4296-4345-89aa-b5211edba587 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:45.928788 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000405d3-4296-4345-89aa-b5211edba587 None admin] PATH_INFO: `/v3/domains` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:45.929738 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000405d3-4296-4345-89aa-b5211edba587 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:17:45.930047 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000405d3-4296-4345-89aa-b5211edba587 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:17:45.931593 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:17:45.932356 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000405d3-4296-4345-89aa-b5211edba587 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:17:45.945804 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 318/649] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Sep 6 04:17:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:17:45.946777 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000405d3-4296-4345-89aa-b5211edba587 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:45.966473 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f205ebf2-5bb4-4718-994d-03af0f50c098 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:46.003007 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1d6eb9-d390-4526-88e6-75911a299f28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:46.004228 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1d6eb9-d390-4526-88e6-75911a299f28 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:46.004509 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1d6eb9-d390-4526-88e6-75911a299f28 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:46.004781 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1d6eb9-d390-4526-88e6-75911a299f28 None admin] PATH_INFO: `/v3/domains` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:46.005467 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1d6eb9-d390-4526-88e6-75911a299f28 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:17:46.005768 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1d6eb9-d390-4526-88e6-75911a299f28 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:17:46.007148 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1d6eb9-d390-4526-88e6-75911a299f28 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:17:46.019921 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 332/650] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Sep 6 04:17:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:17:46.020723 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1d6eb9-d390-4526-88e6-75911a299f28 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:46.058673 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000405d3-4296-4345-89aa-b5211edba587 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:46.094270 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1bed1df-2bd7-42ee-b3c5-dc73d3f7c5ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:46.094902 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bed1df-2bd7-42ee-b3c5-dc73d3f7c5ab None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:46.095178 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bed1df-2bd7-42ee-b3c5-dc73d3f7c5ab None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:46.095446 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bed1df-2bd7-42ee-b3c5-dc73d3f7c5ab None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:46.106024 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1bed1df-2bd7-42ee-b3c5-dc73d3f7c5ab None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:17:46.107455 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1bed1df-2bd7-42ee-b3c5-dc73d3f7c5ab None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:17:46.122604 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-b1bed1df-2bd7-42ee-b3c5-dc73d3f7c5ab None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 04:17:46.122604 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 04:17:46.122604 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 04:17:46.122604 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 04:17:46.122604 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 04:17:46.122604 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 04:17:46.122604 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 04:17:46.122604 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 04:17:46.122604 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 04:17:46.122604 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 04:17:46.122604 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 04:17:46.122604 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 04:17:46.122604 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 04:17:46.122604 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 04:17:46.122604 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 04:17:46.122604 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 04:17:46.123563 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 04:17:46.123563 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 04:17:46.123563 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 04:17:46.123563 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 04:17:46.123563 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 04:17:46.123563 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 04:17:46.123563 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 04:17:46.123563 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 04:17:46.123563 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 04:17:46.123563 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 04:17:46.123563 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 04:17:46.123563 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 04:17:46.123563 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 04:17:46.123563 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 04:17:46.123563 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 04:17:46.123563 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 04:17:46.123563 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 04:17:46.124451 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 04:17:46.124451 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 04:17:46.124451 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 04:17:46.124451 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 319/651] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Sep 6 04:17:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 04:17:46.124451 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1bed1df-2bd7-42ee-b3c5-dc73d3f7c5ab None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:46.137693 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1d6eb9-d390-4526-88e6-75911a299f28 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:46.173239 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:17:46.173906 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:17:46.180596 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75deefbc-495d-45d5-9c2f-8a086da7e665 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:46.181432 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75deefbc-495d-45d5-9c2f-8a086da7e665 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:46.185120 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75deefbc-495d-45d5-9c2f-8a086da7e665 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:46.185120 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75deefbc-495d-45d5-9c2f-8a086da7e665 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:46.195768 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75deefbc-495d-45d5-9c2f-8a086da7e665 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:17:46.195768 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75deefbc-495d-45d5-9c2f-8a086da7e665 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:17:46.210752 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-75deefbc-495d-45d5-9c2f-8a086da7e665 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 04:17:46.210752 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 04:17:46.210752 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 04:17:46.210752 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 04:17:46.210752 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 04:17:46.210752 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 04:17:46.210752 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 04:17:46.210752 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 04:17:46.210752 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 04:17:46.210752 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 04:17:46.210752 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 04:17:46.210752 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 04:17:46.210752 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 04:17:46.210752 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 04:17:46.210752 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 04:17:46.210752 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 04:17:46.211735 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 04:17:46.211735 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 04:17:46.211735 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 04:17:46.211735 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 04:17:46.211735 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 04:17:46.211735 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 04:17:46.211735 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 04:17:46.211735 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 04:17:46.211735 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 04:17:46.211735 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 04:17:46.211735 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 04:17:46.211735 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 04:17:46.211735 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 04:17:46.211735 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 04:17:46.211735 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 04:17:46.211735 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 04:17:46.211735 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 04:17:46.212615 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 04:17:46.212615 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 04:17:46.212615 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 04:17:46.212615 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 333/652] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Sep 6 04:17:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 04:17:46.212615 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75deefbc-495d-45d5-9c2f-8a086da7e665 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:46.248925 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1bed1df-2bd7-42ee-b3c5-dc73d3f7c5ab None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:46.282597 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4327a865-9ee4-4ab6-a39a-02bbafe9bfe6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:46.283234 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4327a865-9ee4-4ab6-a39a-02bbafe9bfe6 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:46.283512 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4327a865-9ee4-4ab6-a39a-02bbafe9bfe6 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:46.283786 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4327a865-9ee4-4ab6-a39a-02bbafe9bfe6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:46.291460 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4327a865-9ee4-4ab6-a39a-02bbafe9bfe6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:17:46.292906 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4327a865-9ee4-4ab6-a39a-02bbafe9bfe6 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:17:46.305152 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-4327a865-9ee4-4ab6-a39a-02bbafe9bfe6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 04:17:46.305152 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 04:17:46.305152 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 04:17:46.305152 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 04:17:46.305152 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 04:17:46.305152 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 04:17:46.305152 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 04:17:46.305152 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 04:17:46.305152 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 04:17:46.305152 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 04:17:46.305152 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 04:17:46.305152 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 04:17:46.305152 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 06 04:17:46.305152 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 06 04:17:46.305152 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 06 04:17:46.306102 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 06 04:17:46.306102 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 04:17:46.306102 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 04:17:46.306102 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 04:17:46.306102 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 04:17:46.306102 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 04:17:46.306102 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 04:17:46.306102 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 04:17:46.306102 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 04:17:46.306102 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 04:17:46.306102 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 04:17:46.306102 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 04:17:46.306102 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 04:17:46.306102 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 04:17:46.306102 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 04:17:46.306102 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 04:17:46.306102 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 04:17:46.306990 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 06 04:17:46.306990 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 06 04:17:46.306990 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 06 04:17:46.306990 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 06 04:17:46.306990 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 06 04:17:46.306990 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 06 04:17:46.306990 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 04:17:46.306990 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 04:17:46.306990 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 320/653] 10.222.0.30 () {56 vars in 1111 bytes} [Tue Sep 6 04:17:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 04:17:46.307471 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4327a865-9ee4-4ab6-a39a-02bbafe9bfe6 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:46.323289 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75deefbc-495d-45d5-9c2f-8a086da7e665 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:46.358272 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35264711-4899-4314-a954-247b64d49847 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:46.359021 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35264711-4899-4314-a954-247b64d49847 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:46.359404 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35264711-4899-4314-a954-247b64d49847 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:46.359750 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35264711-4899-4314-a954-247b64d49847 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:46.369075 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35264711-4899-4314-a954-247b64d49847 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:17:46.370645 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35264711-4899-4314-a954-247b64d49847 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:17:46.383180 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-35264711-4899-4314-a954-247b64d49847 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 04:17:46.383180 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 04:17:46.383180 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 04:17:46.383180 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 04:17:46.383180 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 04:17:46.383180 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 04:17:46.383180 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 04:17:46.383180 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 04:17:46.383180 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 04:17:46.383180 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 04:17:46.383180 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 04:17:46.383180 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 04:17:46.383180 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 06 04:17:46.383180 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 06 04:17:46.383180 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 06 04:17:46.384250 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 06 04:17:46.384250 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 04:17:46.384250 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 04:17:46.384250 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 04:17:46.384250 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 04:17:46.384250 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 04:17:46.384250 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 04:17:46.384250 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 04:17:46.384250 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 04:17:46.384250 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 04:17:46.384250 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 04:17:46.384250 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 04:17:46.384250 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 04:17:46.384250 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 04:17:46.384250 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 04:17:46.384250 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 04:17:46.384250 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 04:17:46.385448 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 06 04:17:46.385448 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 06 04:17:46.385448 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 06 04:17:46.385448 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 06 04:17:46.385448 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 06 04:17:46.385448 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 06 04:17:46.385448 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 04:17:46.385448 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 04:17:46.389070 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 334/654] 10.222.0.30 () {56 vars in 1111 bytes} [Tue Sep 6 04:17:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 04:17:46.389920 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35264711-4899-4314-a954-247b64d49847 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:46.418940 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:17:46.422416 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4327a865-9ee4-4ab6-a39a-02bbafe9bfe6 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:46.442574 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:17:46.461258 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea2619b-135e-4c96-a503-ae37bc706650 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:46.461901 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea2619b-135e-4c96-a503-ae37bc706650 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:46.462181 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea2619b-135e-4c96-a503-ae37bc706650 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:46.462453 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea2619b-135e-4c96-a503-ae37bc706650 None admin] PATH_INFO: `/v3/domains` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:46.463105 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea2619b-135e-4c96-a503-ae37bc706650 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:17:46.463405 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea2619b-135e-4c96-a503-ae37bc706650 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:17:46.464767 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea2619b-135e-4c96-a503-ae37bc706650 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:17:46.481129 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 321/655] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Sep 6 04:17:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:17:46.481129 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea2619b-135e-4c96-a503-ae37bc706650 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:46.508987 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35264711-4899-4314-a954-247b64d49847 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:46.543473 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8283a0b0-ecde-47ef-bccf-25230e7390a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:46.544091 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8283a0b0-ecde-47ef-bccf-25230e7390a1 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:46.544370 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8283a0b0-ecde-47ef-bccf-25230e7390a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:46.544646 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8283a0b0-ecde-47ef-bccf-25230e7390a1 None admin] PATH_INFO: `/v3/domains` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:46.545292 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8283a0b0-ecde-47ef-bccf-25230e7390a1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:17:46.545620 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8283a0b0-ecde-47ef-bccf-25230e7390a1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:17:46.546994 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8283a0b0-ecde-47ef-bccf-25230e7390a1 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:17:46.559688 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 335/656] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Sep 6 04:17:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:17:46.560656 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8283a0b0-ecde-47ef-bccf-25230e7390a1 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:46.580252 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea2619b-135e-4c96-a503-ae37bc706650 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:46.613964 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7878348c-fc0e-4f60-97fa-844da2682835 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:46.614565 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7878348c-fc0e-4f60-97fa-844da2682835 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:46.614832 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7878348c-fc0e-4f60-97fa-844da2682835 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:46.615102 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7878348c-fc0e-4f60-97fa-844da2682835 None admin] PATH_INFO: `/v3/projects` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:46.615823 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7878348c-fc0e-4f60-97fa-844da2682835 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:17:46.616126 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7878348c-fc0e-4f60-97fa-844da2682835 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:17:46.617578 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7878348c-fc0e-4f60-97fa-844da2682835 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:17:46.629080 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 322/657] 10.222.0.30 () {56 vars in 1148 bytes} [Tue Sep 6 04:17:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:17:46.629905 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7878348c-fc0e-4f60-97fa-844da2682835 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:46.648818 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:17:46.662315 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8283a0b0-ecde-47ef-bccf-25230e7390a1 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:46.665742 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:17:46.697677 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba776e1-23d0-4b54-a12e-853dcf689f01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:46.698216 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba776e1-23d0-4b54-a12e-853dcf689f01 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:46.698495 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba776e1-23d0-4b54-a12e-853dcf689f01 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:46.698768 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba776e1-23d0-4b54-a12e-853dcf689f01 None admin] PATH_INFO: `/v3/projects` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:46.699458 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba776e1-23d0-4b54-a12e-853dcf689f01 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:17:46.699764 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba776e1-23d0-4b54-a12e-853dcf689f01 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:17:46.701216 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba776e1-23d0-4b54-a12e-853dcf689f01 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:17:46.713943 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 336/658] 10.222.0.30 () {56 vars in 1148 bytes} [Tue Sep 6 04:17:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:17:46.715203 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85dd7735-7fda-4a22-a1ca-047a75251353 None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:46.715555 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85dd7735-7fda-4a22-a1ca-047a75251353 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:46.715905 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85dd7735-7fda-4a22-a1ca-047a75251353 None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:46.717389 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 337/659] 10.222.0.30 () {54 vars in 927 bytes} [Tue Sep 6 04:17:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 04:17:46.721901 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85dd7735-7fda-4a22-a1ca-047a75251353 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:46.730876 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7878348c-fc0e-4f60-97fa-844da2682835 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:46.764282 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9aec28-7433-441a-93f2-5179d4ee0eed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:46.764912 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9aec28-7433-441a-93f2-5179d4ee0eed None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:46.765190 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9aec28-7433-441a-93f2-5179d4ee0eed None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:46.765490 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9aec28-7433-441a-93f2-5179d4ee0eed None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:46.773556 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b9aec28-7433-441a-93f2-5179d4ee0eed None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:17:46.774998 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b9aec28-7433-441a-93f2-5179d4ee0eed None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:17:46.785970 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-9b9aec28-7433-441a-93f2-5179d4ee0eed None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 06 04:17:46.785970 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 04:17:46.785970 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 04:17:46.785970 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 04:17:46.785970 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 04:17:46.785970 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 04:17:46.785970 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 04:17:46.785970 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 04:17:46.785970 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 04:17:46.785970 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 04:17:46.785970 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 04:17:46.785970 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 04:17:46.785970 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 06 04:17:46.785970 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 06 04:17:46.785970 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 06 04:17:46.785970 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 06 04:17:46.787078 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 04:17:46.787078 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 04:17:46.787078 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 06 04:17:46.787078 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 04:17:46.787078 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 06 04:17:46.787078 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 04:17:46.787078 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 04:17:46.787078 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 04:17:46.787078 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 04:17:46.787078 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 04:17:46.787078 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 04:17:46.787078 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 04:17:46.787078 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 04:17:46.787078 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 04:17:46.787078 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 04:17:46.787078 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 04:17:46.787078 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 04:17:46.787078 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 04:17:46.788054 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 04:17:46.788054 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 04:17:46.788054 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 06 04:17:46.788054 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 06 04:17:46.788054 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 06 04:17:46.788054 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 06 04:17:46.788054 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 06 04:17:46.788054 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 06 04:17:46.788054 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 06 04:17:46.788054 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 04:17:46.788054 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 323/660] 10.222.0.30 () {56 vars in 1099 bytes} [Tue Sep 6 04:17:46 2022] GET /identity/v3/users/cinder => generated 84 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 04:17:46.788054 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9aec28-7433-441a-93f2-5179d4ee0eed None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:46.828859 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85dd7735-7fda-4a22-a1ca-047a75251353 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:46.861984 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217b10e7-456a-4f2b-88d9-932cc3e7f496 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:46.862746 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217b10e7-456a-4f2b-88d9-932cc3e7f496 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:46.863066 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217b10e7-456a-4f2b-88d9-932cc3e7f496 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:46.863338 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217b10e7-456a-4f2b-88d9-932cc3e7f496 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:46.865071 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:17:46.870927 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-217b10e7-456a-4f2b-88d9-932cc3e7f496 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:17:46.872393 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-217b10e7-456a-4f2b-88d9-932cc3e7f496 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:17:46.884132 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-217b10e7-456a-4f2b-88d9-932cc3e7f496 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 06 04:17:46.884132 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 04:17:46.884132 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 04:17:46.884132 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 04:17:46.884132 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 04:17:46.884132 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 04:17:46.884132 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 04:17:46.884132 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 04:17:46.884132 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 04:17:46.884132 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 04:17:46.884132 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 04:17:46.884132 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 04:17:46.884132 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 06 04:17:46.884132 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 06 04:17:46.884132 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 06 04:17:46.884132 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 06 04:17:46.885277 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 04:17:46.885277 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 04:17:46.885277 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 06 04:17:46.885277 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 04:17:46.885277 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 06 04:17:46.885277 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 04:17:46.885277 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 04:17:46.885277 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 04:17:46.885277 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 04:17:46.885277 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 04:17:46.885277 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 04:17:46.885277 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 04:17:46.885277 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 04:17:46.885277 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 04:17:46.885277 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 04:17:46.885277 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 04:17:46.885277 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 04:17:46.885277 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 04:17:46.886419 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 04:17:46.886419 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 04:17:46.886419 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 06 04:17:46.886419 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 06 04:17:46.886419 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 06 04:17:46.886419 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 06 04:17:46.886419 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 06 04:17:46.886419 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 06 04:17:46.886419 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 06 04:17:46.886419 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 04:17:46.886419 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 338/661] 10.222.0.30 () {56 vars in 1102 bytes} [Tue Sep 6 04:17:46 2022] GET /identity/v3/users/neutron => generated 85 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 04:17:46.887213 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9aec28-7433-441a-93f2-5179d4ee0eed None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:46.888174 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217b10e7-456a-4f2b-88d9-932cc3e7f496 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:46.913539 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:17:46.921389 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34396e9-573d-4210-8036-dba050d34a58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:46.921999 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34396e9-573d-4210-8036-dba050d34a58 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:46.922278 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34396e9-573d-4210-8036-dba050d34a58 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:46.922550 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34396e9-573d-4210-8036-dba050d34a58 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:46.923848 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34396e9-573d-4210-8036-dba050d34a58 None admin] RBAC: Adding query filter params (role.id=83cf0a194f8d45e792411e2d9afdfe50, scope.project.id=b707733ce65c42edbc1fd95ad0a1c06c, user.id=9db138b77afa4b068244b8ef715404fe) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:17:46.923848 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34396e9-573d-4210-8036-dba050d34a58 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:17:46.924965 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34396e9-573d-4210-8036-dba050d34a58 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:17:46.930952 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 324/662] 10.222.0.30 () {56 vars in 1374 bytes} [Tue Sep 6 04:17:46 2022] GET /identity/v3/role_assignments?user.id=9db138b77afa4b068244b8ef715404fe&scope.project.id=b707733ce65c42edbc1fd95ad0a1c06c&role.id=83cf0a194f8d45e792411e2d9afdfe50 => generated 608 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:17:46.932100 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb57064-392c-46a2-a0bf-1502feb98eb3 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:46.932387 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb57064-392c-46a2-a0bf-1502feb98eb3 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:46.932664 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb57064-392c-46a2-a0bf-1502feb98eb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:46.959694 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-3bb57064-392c-46a2-a0bf-1502feb98eb3 None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:17:46.961169 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-3bb57064-392c-46a2-a0bf-1502feb98eb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:17:47.005488 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217b10e7-456a-4f2b-88d9-932cc3e7f496 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:47.042728 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 325/663] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Sep 6 04:17:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 04:17:47.043594 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb57064-392c-46a2-a0bf-1502feb98eb3 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:47.062946 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32fa20f0-901b-42c7-8ee7-d8fbea9794d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:47.064343 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fa20f0-901b-42c7-8ee7-d8fbea9794d9 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:47.064888 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fa20f0-901b-42c7-8ee7-d8fbea9794d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:47.065395 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fa20f0-901b-42c7-8ee7-d8fbea9794d9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:47.066969 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32fa20f0-901b-42c7-8ee7-d8fbea9794d9 None admin] RBAC: Adding query filter params (role.id=83cf0a194f8d45e792411e2d9afdfe50, scope.project.id=b707733ce65c42edbc1fd95ad0a1c06c, user.id=ae5c31fb1d0342c0937bbc83a415113d) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:17:47.068104 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32fa20f0-901b-42c7-8ee7-d8fbea9794d9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:17:47.070125 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32fa20f0-901b-42c7-8ee7-d8fbea9794d9 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:17:47.078999 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 339/664] 10.222.0.30 () {56 vars in 1374 bytes} [Tue Sep 6 04:17:46 2022] GET /identity/v3/role_assignments?user.id=ae5c31fb1d0342c0937bbc83a415113d&scope.project.id=b707733ce65c42edbc1fd95ad0a1c06c&role.id=83cf0a194f8d45e792411e2d9afdfe50 => generated 608 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:17:47.084982 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32fa20f0-901b-42c7-8ee7-d8fbea9794d9 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:47.136028 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:17:47.182170 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb57064-392c-46a2-a0bf-1502feb98eb3 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:47.221738 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:17:47.225675 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeaf2c91-f8b7-4bf6-8887-5a0403a54cd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:47.226456 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeaf2c91-f8b7-4bf6-8887-5a0403a54cd4 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:47.226829 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeaf2c91-f8b7-4bf6-8887-5a0403a54cd4 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:47.227167 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeaf2c91-f8b7-4bf6-8887-5a0403a54cd4 None admin] PATH_INFO: `/v3/users` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:47.227973 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeaf2c91-f8b7-4bf6-8887-5a0403a54cd4 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:17:47.228408 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeaf2c91-f8b7-4bf6-8887-5a0403a54cd4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:17:47.229957 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeaf2c91-f8b7-4bf6-8887-5a0403a54cd4 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:17:47.236382 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32fa20f0-901b-42c7-8ee7-d8fbea9794d9 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:47.251544 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 326/665] 10.222.0.30 () {56 vars in 1137 bytes} [Tue Sep 6 04:17:47 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:17:47.258382 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeaf2c91-f8b7-4bf6-8887-5a0403a54cd4 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:47.288997 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3420492e-260e-41f1-a2ff-d0f3ac3521a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:47.289950 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3420492e-260e-41f1-a2ff-d0f3ac3521a2 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:47.290473 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3420492e-260e-41f1-a2ff-d0f3ac3521a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:47.290912 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3420492e-260e-41f1-a2ff-d0f3ac3521a2 None admin] PATH_INFO: `/v3/users` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:47.291898 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3420492e-260e-41f1-a2ff-d0f3ac3521a2 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:17:47.292352 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3420492e-260e-41f1-a2ff-d0f3ac3521a2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:17:47.294174 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3420492e-260e-41f1-a2ff-d0f3ac3521a2 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:17:47.319736 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 340/666] 10.222.0.30 () {56 vars in 1139 bytes} [Tue Sep 6 04:17:47 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:17:47.325422 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3420492e-260e-41f1-a2ff-d0f3ac3521a2 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:47.392150 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeaf2c91-f8b7-4bf6-8887-5a0403a54cd4 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:47.428982 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:17:47.438392 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b910d165-e0a8-4348-bcca-a81da458dd76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:47.439081 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b910d165-e0a8-4348-bcca-a81da458dd76 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:47.439373 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b910d165-e0a8-4348-bcca-a81da458dd76 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:47.439645 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b910d165-e0a8-4348-bcca-a81da458dd76 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:47.470629 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3420492e-260e-41f1-a2ff-d0f3ac3521a2 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:47.510180 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420ef9b3-997f-430b-97c6-79da61eb202e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:47.510925 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420ef9b3-997f-430b-97c6-79da61eb202e None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:47.511306 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420ef9b3-997f-430b-97c6-79da61eb202e None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:47.511681 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420ef9b3-997f-430b-97c6-79da61eb202e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:47.528590 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:17:47.536639 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b910d165-e0a8-4348-bcca-a81da458dd76 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:17:47.538781 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b910d165-e0a8-4348-bcca-a81da458dd76 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:17:47.589099 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420ef9b3-997f-430b-97c6-79da61eb202e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:17:47.590760 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420ef9b3-997f-430b-97c6-79da61eb202e None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:17:47.679242 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 327/667] 10.222.0.30 () {58 vars in 1282 bytes} [Tue Sep 6 04:17:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 422 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 06 04:17:47.680298 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b910d165-e0a8-4348-bcca-a81da458dd76 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:47.708986 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 341/668] 10.222.0.30 () {58 vars in 1282 bytes} [Tue Sep 6 04:17:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 06 04:17:47.708986 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420ef9b3-997f-430b-97c6-79da61eb202e None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:47.715831 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:17:47.770397 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:17:47.810467 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b910d165-e0a8-4348-bcca-a81da458dd76 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:47.826041 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420ef9b3-997f-430b-97c6-79da61eb202e None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:47.862244 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08441d82-f56e-4f3f-8455-97005312add3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:47.863945 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08441d82-f56e-4f3f-8455-97005312add3 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:47.864539 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08441d82-f56e-4f3f-8455-97005312add3 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:47.864959 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08441d82-f56e-4f3f-8455-97005312add3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:47.867622 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687e5351-651b-4cc6-8f24-4c31c761a1d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:47.868363 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687e5351-651b-4cc6-8f24-4c31c761a1d4 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:47.868745 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687e5351-651b-4cc6-8f24-4c31c761a1d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:47.869119 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687e5351-651b-4cc6-8f24-4c31c761a1d4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:47.878032 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08441d82-f56e-4f3f-8455-97005312add3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 04:17:47.881402 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687e5351-651b-4cc6-8f24-4c31c761a1d4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:17:47.882985 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687e5351-651b-4cc6-8f24-4c31c761a1d4 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:17:47.885283 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-08441d82-f56e-4f3f-8455-97005312add3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 06 04:17:47.885283 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 04:17:47.885283 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 04:17:47.885283 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 04:17:47.885283 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 04:17:47.885283 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 04:17:47.885283 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 04:17:47.885283 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 04:17:47.885283 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 04:17:47.885283 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 04:17:47.885283 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 04:17:47.885283 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 04:17:47.885283 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 04:17:47.885283 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 04:17:47.885283 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 04:17:47.885283 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 04:17:47.886433 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 04:17:47.886433 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 04:17:47.886433 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 04:17:47.886433 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 04:17:47.886433 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 04:17:47.886433 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 04:17:47.886433 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 04:17:47.886433 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 04:17:47.886433 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 04:17:47.886433 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 04:17:47.886433 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 04:17:47.886433 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 04:17:47.886433 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 04:17:47.886433 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 04:17:47.886433 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 04:17:47.886433 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 04:17:47.887438 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 04:17:47.887438 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 04:17:47.887438 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 04:17:47.887438 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 04:17:47.887438 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 04:17:47.887438 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 04:17:47.887438 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 04:17:47.887438 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 04:17:47.887438 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 04:17:47.887438 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 04:17:47.887438 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 06 04:17:47.887438 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 04:17:47.890168 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 328/669] 10.222.0.30 () {56 vars in 1102 bytes} [Tue Sep 6 04:17:47 2022] GET /identity/v3/roles/service => generated 85 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 04:17:47.891155 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08441d82-f56e-4f3f-8455-97005312add3 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:47.900972 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-687e5351-651b-4cc6-8f24-4c31c761a1d4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 04:17:47.900972 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 04:17:47.900972 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 04:17:47.900972 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 04:17:47.900972 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 04:17:47.900972 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 04:17:47.900972 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 04:17:47.900972 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 04:17:47.900972 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 04:17:47.900972 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 04:17:47.900972 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 04:17:47.900972 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 04:17:47.900972 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 04:17:47.900972 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 04:17:47.900972 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 04:17:47.900972 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 04:17:47.902139 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 04:17:47.902139 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 04:17:47.902139 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 04:17:47.902139 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 04:17:47.902139 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 04:17:47.902139 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 04:17:47.902139 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 04:17:47.902139 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 04:17:47.902139 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 04:17:47.902139 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 04:17:47.902139 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 04:17:47.902139 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 04:17:47.902139 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 04:17:47.902139 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 04:17:47.902139 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 04:17:47.902139 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 04:17:47.902139 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 04:17:47.903206 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 04:17:47.903206 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 04:17:47.903206 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 04:17:47.904546 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 342/670] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Sep 6 04:17:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 04:17:47.905519 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687e5351-651b-4cc6-8f24-4c31c761a1d4 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:48.012277 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:17:48.016245 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687e5351-651b-4cc6-8f24-4c31c761a1d4 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:48.024149 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:17:48.037324 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08441d82-f56e-4f3f-8455-97005312add3 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:48.052278 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48494e0b-cf52-447a-ae82-325995ebff39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:48.053031 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48494e0b-cf52-447a-ae82-325995ebff39 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:48.053439 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48494e0b-cf52-447a-ae82-325995ebff39 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:48.053813 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48494e0b-cf52-447a-ae82-325995ebff39 None admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:48.054600 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48494e0b-cf52-447a-ae82-325995ebff39 None admin] RBAC: Adding query filter params (name=service) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:17:48.055005 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48494e0b-cf52-447a-ae82-325995ebff39 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:17:48.056470 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48494e0b-cf52-447a-ae82-325995ebff39 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:17:48.065205 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 343/671] 10.222.0.30 () {56 vars in 1103 bytes} [Tue Sep 6 04:17:47 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:17:48.066412 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48494e0b-cf52-447a-ae82-325995ebff39 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:48.078325 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e32a75-567e-4044-bb4c-96181852238f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:48.081986 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e32a75-567e-4044-bb4c-96181852238f None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:48.081986 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e32a75-567e-4044-bb4c-96181852238f None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:48.081986 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e32a75-567e-4044-bb4c-96181852238f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:48.092239 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e32a75-567e-4044-bb4c-96181852238f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:17:48.093990 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e32a75-567e-4044-bb4c-96181852238f None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:17:48.110405 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-d4e32a75-567e-4044-bb4c-96181852238f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 04:17:48.110405 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 04:17:48.110405 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 04:17:48.110405 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 04:17:48.110405 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 04:17:48.110405 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 04:17:48.110405 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 04:17:48.110405 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 04:17:48.110405 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 04:17:48.110405 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 04:17:48.110405 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 04:17:48.110405 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 04:17:48.110405 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 04:17:48.110405 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 04:17:48.110405 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 04:17:48.110405 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 04:17:48.111539 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 04:17:48.111539 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 04:17:48.111539 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 04:17:48.111539 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 04:17:48.111539 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 04:17:48.111539 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 04:17:48.111539 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 04:17:48.111539 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 04:17:48.111539 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 04:17:48.111539 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 04:17:48.111539 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 04:17:48.111539 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 04:17:48.111539 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 04:17:48.111539 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 04:17:48.111539 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 04:17:48.111539 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 04:17:48.111539 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 04:17:48.113251 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 04:17:48.113251 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 04:17:48.113251 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 04:17:48.115074 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 329/672] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Sep 6 04:17:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 04:17:48.116199 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e32a75-567e-4044-bb4c-96181852238f None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:48.183033 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48494e0b-cf52-447a-ae82-325995ebff39 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:48.218886 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f25ebe6-7492-4ea8-b357-d2bf5b2cf0c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:48.219670 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f25ebe6-7492-4ea8-b357-d2bf5b2cf0c7 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:48.220061 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f25ebe6-7492-4ea8-b357-d2bf5b2cf0c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:48.220443 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f25ebe6-7492-4ea8-b357-d2bf5b2cf0c7 None admin] PATH_INFO: `/v3/domains` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:48.221208 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f25ebe6-7492-4ea8-b357-d2bf5b2cf0c7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:17:48.221660 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f25ebe6-7492-4ea8-b357-d2bf5b2cf0c7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:17:48.223170 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f25ebe6-7492-4ea8-b357-d2bf5b2cf0c7 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:17:48.235870 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 344/673] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Sep 6 04:17:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:17:48.236829 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f25ebe6-7492-4ea8-b357-d2bf5b2cf0c7 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:48.257167 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e32a75-567e-4044-bb4c-96181852238f None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:48.260763 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:17:48.296621 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada97a07-8219-48e7-b7a7-afd5d8d409d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:48.297691 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada97a07-8219-48e7-b7a7-afd5d8d409d7 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:48.298114 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada97a07-8219-48e7-b7a7-afd5d8d409d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:48.298529 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada97a07-8219-48e7-b7a7-afd5d8d409d7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:48.312038 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada97a07-8219-48e7-b7a7-afd5d8d409d7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:17:48.313868 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada97a07-8219-48e7-b7a7-afd5d8d409d7 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:17:48.316404 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:17:48.331003 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-ada97a07-8219-48e7-b7a7-afd5d8d409d7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 04:17:48.331003 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 04:17:48.331003 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 04:17:48.331003 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 04:17:48.331003 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 04:17:48.331003 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 04:17:48.331003 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 04:17:48.331003 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 04:17:48.331003 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 04:17:48.331003 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 04:17:48.331003 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 04:17:48.331003 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 04:17:48.331003 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 04:17:48.331003 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 04:17:48.331003 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 04:17:48.331003 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 04:17:48.333321 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 04:17:48.333321 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 04:17:48.333321 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 04:17:48.333321 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 04:17:48.333321 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 04:17:48.333321 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 04:17:48.333321 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 04:17:48.333321 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 04:17:48.333321 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 04:17:48.333321 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 04:17:48.333321 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 04:17:48.333321 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 04:17:48.333321 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 04:17:48.333321 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 04:17:48.333321 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 04:17:48.333321 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 04:17:48.333321 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 04:17:48.334362 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 04:17:48.334362 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 04:17:48.334362 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 04:17:48.335881 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 330/674] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Sep 6 04:17:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 04:17:48.336956 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada97a07-8219-48e7-b7a7-afd5d8d409d7 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:48.348978 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f25ebe6-7492-4ea8-b357-d2bf5b2cf0c7 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:48.397107 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0b9b0a-5923-47a8-afe2-e11ed5110c56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:48.397780 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0b9b0a-5923-47a8-afe2-e11ed5110c56 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:48.398062 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0b9b0a-5923-47a8-afe2-e11ed5110c56 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:48.398337 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0b9b0a-5923-47a8-afe2-e11ed5110c56 None admin] PATH_INFO: `/v3/domains` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:48.398988 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa0b9b0a-5923-47a8-afe2-e11ed5110c56 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:17:48.399307 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa0b9b0a-5923-47a8-afe2-e11ed5110c56 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:17:48.401424 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa0b9b0a-5923-47a8-afe2-e11ed5110c56 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:17:48.421740 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 345/675] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Sep 6 04:17:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:17:48.421740 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0b9b0a-5923-47a8-afe2-e11ed5110c56 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:48.460991 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada97a07-8219-48e7-b7a7-afd5d8d409d7 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:48.499431 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e97ac45-b4ab-4568-8958-be85b2c0d5e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:48.500507 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e97ac45-b4ab-4568-8958-be85b2c0d5e2 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:48.501013 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e97ac45-b4ab-4568-8958-be85b2c0d5e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:48.501599 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e97ac45-b4ab-4568-8958-be85b2c0d5e2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:48.510893 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e97ac45-b4ab-4568-8958-be85b2c0d5e2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:17:48.513619 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e97ac45-b4ab-4568-8958-be85b2c0d5e2 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:17:48.514425 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:17:48.534264 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0b9b0a-5923-47a8-afe2-e11ed5110c56 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:48.534724 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-4e97ac45-b4ab-4568-8958-be85b2c0d5e2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 04:17:48.534724 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 04:17:48.534724 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 04:17:48.534724 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 04:17:48.534724 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 04:17:48.534724 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 04:17:48.534724 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 04:17:48.534724 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 04:17:48.534724 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 04:17:48.534724 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 04:17:48.534724 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 04:17:48.534724 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 04:17:48.534724 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 06 04:17:48.534724 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 06 04:17:48.534724 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 06 04:17:48.536434 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 06 04:17:48.536434 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 04:17:48.536434 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 04:17:48.536434 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 04:17:48.536434 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 04:17:48.536434 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 04:17:48.536434 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 04:17:48.536434 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 04:17:48.536434 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 04:17:48.536434 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 04:17:48.536434 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 04:17:48.536434 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 04:17:48.536434 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 04:17:48.536434 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 04:17:48.536434 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 04:17:48.536434 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 04:17:48.536434 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 04:17:48.538137 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 06 04:17:48.538137 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 06 04:17:48.538137 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 06 04:17:48.538137 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 06 04:17:48.538137 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 06 04:17:48.538137 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 06 04:17:48.538137 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 04:17:48.538137 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 04:17:48.542820 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 331/676] 10.222.0.30 () {56 vars in 1111 bytes} [Tue Sep 6 04:17:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 04:17:48.544179 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e97ac45-b4ab-4568-8958-be85b2c0d5e2 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:48.573648 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32acf835-c806-497a-8a0d-89cd7c9e2db8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:48.574274 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32acf835-c806-497a-8a0d-89cd7c9e2db8 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:48.574552 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32acf835-c806-497a-8a0d-89cd7c9e2db8 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:48.574826 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32acf835-c806-497a-8a0d-89cd7c9e2db8 None admin] PATH_INFO: `/v3/domains` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:48.575904 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32acf835-c806-497a-8a0d-89cd7c9e2db8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:17:48.575904 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32acf835-c806-497a-8a0d-89cd7c9e2db8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:17:48.577218 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32acf835-c806-497a-8a0d-89cd7c9e2db8 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:17:48.589016 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 346/677] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Sep 6 04:17:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:17:48.589897 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32acf835-c806-497a-8a0d-89cd7c9e2db8 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:48.613743 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:17:48.666001 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e97ac45-b4ab-4568-8958-be85b2c0d5e2 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:48.699311 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32acf835-c806-497a-8a0d-89cd7c9e2db8 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:48.705080 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26dc1626-3d00-49fd-9796-36159af41e7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:48.706455 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26dc1626-3d00-49fd-9796-36159af41e7e None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:48.706966 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26dc1626-3d00-49fd-9796-36159af41e7e None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:48.707413 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26dc1626-3d00-49fd-9796-36159af41e7e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:48.715109 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26dc1626-3d00-49fd-9796-36159af41e7e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:17:48.716960 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26dc1626-3d00-49fd-9796-36159af41e7e None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:17:48.729817 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-26dc1626-3d00-49fd-9796-36159af41e7e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 04:17:48.729817 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 04:17:48.729817 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 04:17:48.729817 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 04:17:48.729817 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 04:17:48.729817 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 04:17:48.729817 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 04:17:48.729817 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 04:17:48.729817 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 04:17:48.729817 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 04:17:48.729817 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 04:17:48.729817 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 04:17:48.729817 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 06 04:17:48.729817 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 06 04:17:48.729817 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 06 04:17:48.730729 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 06 04:17:48.730729 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 04:17:48.730729 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 04:17:48.730729 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 04:17:48.730729 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 04:17:48.730729 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 04:17:48.730729 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 04:17:48.730729 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 04:17:48.730729 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 04:17:48.730729 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 04:17:48.730729 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 04:17:48.730729 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 04:17:48.730729 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 04:17:48.730729 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 04:17:48.730729 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 04:17:48.730729 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 04:17:48.730729 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 04:17:48.731604 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 06 04:17:48.731604 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 06 04:17:48.731604 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 06 04:17:48.731604 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 06 04:17:48.731604 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 06 04:17:48.731604 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 06 04:17:48.731604 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 04:17:48.731604 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 04:17:48.733583 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 332/678] 10.222.0.30 () {56 vars in 1111 bytes} [Tue Sep 6 04:17:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 04:17:48.735516 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26dc1626-3d00-49fd-9796-36159af41e7e None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:48.735986 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134d1bf1-94b2-4b3a-a3c8-dcaaa4b00fb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:48.740167 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134d1bf1-94b2-4b3a-a3c8-dcaaa4b00fb7 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:48.740167 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134d1bf1-94b2-4b3a-a3c8-dcaaa4b00fb7 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:48.740167 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134d1bf1-94b2-4b3a-a3c8-dcaaa4b00fb7 None admin] PATH_INFO: `/v3/projects` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:48.740167 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134d1bf1-94b2-4b3a-a3c8-dcaaa4b00fb7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:17:48.740167 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134d1bf1-94b2-4b3a-a3c8-dcaaa4b00fb7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:17:48.740765 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134d1bf1-94b2-4b3a-a3c8-dcaaa4b00fb7 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:17:48.751072 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:17:48.755606 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 347/679] 10.222.0.30 () {56 vars in 1148 bytes} [Tue Sep 6 04:17:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:17:48.756552 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134d1bf1-94b2-4b3a-a3c8-dcaaa4b00fb7 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:48.862566 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26dc1626-3d00-49fd-9796-36159af41e7e None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:48.863773 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134d1bf1-94b2-4b3a-a3c8-dcaaa4b00fb7 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:48.881247 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:17:48.900805 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9466f8-08a9-4040-a0f6-a55b354fafaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:48.907155 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9466f8-08a9-4040-a0f6-a55b354fafaf None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:48.907155 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9466f8-08a9-4040-a0f6-a55b354fafaf None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:48.907155 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9466f8-08a9-4040-a0f6-a55b354fafaf None admin] PATH_INFO: `/v3/projects` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:48.907155 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9466f8-08a9-4040-a0f6-a55b354fafaf None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:17:48.907155 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9466f8-08a9-4040-a0f6-a55b354fafaf None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:17:48.907155 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9466f8-08a9-4040-a0f6-a55b354fafaf None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:17:48.908742 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d227a903-4197-43d3-9181-7a7843899e70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:48.910149 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d227a903-4197-43d3-9181-7a7843899e70 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:48.910588 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d227a903-4197-43d3-9181-7a7843899e70 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:48.910998 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d227a903-4197-43d3-9181-7a7843899e70 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:48.923872 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 348/680] 10.222.0.30 () {56 vars in 1148 bytes} [Tue Sep 6 04:17:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:17:48.923872 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9466f8-08a9-4040-a0f6-a55b354fafaf None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:48.927355 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d227a903-4197-43d3-9181-7a7843899e70 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:17:48.935025 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d227a903-4197-43d3-9181-7a7843899e70 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:17:48.952816 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-d227a903-4197-43d3-9181-7a7843899e70 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 06 04:17:48.952816 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 04:17:48.952816 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 04:17:48.952816 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 04:17:48.952816 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 04:17:48.952816 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 04:17:48.952816 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 04:17:48.952816 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 04:17:48.952816 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 04:17:48.952816 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 04:17:48.952816 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 04:17:48.952816 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 04:17:48.952816 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 06 04:17:48.952816 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 06 04:17:48.952816 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 06 04:17:48.952816 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 06 04:17:48.954097 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 04:17:48.954097 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 04:17:48.954097 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 06 04:17:48.954097 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 04:17:48.954097 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 06 04:17:48.954097 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 04:17:48.954097 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 04:17:48.954097 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 04:17:48.954097 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 04:17:48.954097 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 04:17:48.954097 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 04:17:48.954097 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 04:17:48.954097 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 04:17:48.954097 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 04:17:48.954097 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 04:17:48.954097 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 04:17:48.954097 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 04:17:48.954097 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 04:17:48.955235 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 04:17:48.955235 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 04:17:48.955235 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 06 04:17:48.955235 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 06 04:17:48.955235 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 06 04:17:48.955235 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 06 04:17:48.955235 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 06 04:17:48.955235 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 06 04:17:48.955235 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 06 04:17:48.955235 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 04:17:48.958826 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 333/681] 10.222.0.30 () {56 vars in 1096 bytes} [Tue Sep 6 04:17:48 2022] GET /identity/v3/users/swift => generated 83 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 04:17:48.959933 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d227a903-4197-43d3-9181-7a7843899e70 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:49.004868 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:17:49.057161 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9466f8-08a9-4040-a0f6-a55b354fafaf None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:49.092598 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d227a903-4197-43d3-9181-7a7843899e70 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:49.100421 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a785bb3-d37d-480d-a7fa-113cceb8816e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:49.101164 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a785bb3-d37d-480d-a7fa-113cceb8816e None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:49.101572 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a785bb3-d37d-480d-a7fa-113cceb8816e None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:49.102469 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a785bb3-d37d-480d-a7fa-113cceb8816e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:49.103318 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a785bb3-d37d-480d-a7fa-113cceb8816e None admin] RBAC: Adding query filter params (role.id=83cf0a194f8d45e792411e2d9afdfe50, scope.project.id=b707733ce65c42edbc1fd95ad0a1c06c, user.id=d09b5ab675634a2eb4b03755743f7d0d) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:17:49.103737 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a785bb3-d37d-480d-a7fa-113cceb8816e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:17:49.105770 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a785bb3-d37d-480d-a7fa-113cceb8816e None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:17:49.115634 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 349/682] 10.222.0.30 () {56 vars in 1374 bytes} [Tue Sep 6 04:17:48 2022] GET /identity/v3/role_assignments?user.id=d09b5ab675634a2eb4b03755743f7d0d&scope.project.id=b707733ce65c42edbc1fd95ad0a1c06c&role.id=83cf0a194f8d45e792411e2d9afdfe50 => generated 608 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:17:49.116600 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a785bb3-d37d-480d-a7fa-113cceb8816e None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:49.137534 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8495498-a035-43d2-b014-b3a93e0e50cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:49.138680 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8495498-a035-43d2-b014-b3a93e0e50cb None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:49.139134 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8495498-a035-43d2-b014-b3a93e0e50cb None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:49.140094 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8495498-a035-43d2-b014-b3a93e0e50cb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:49.141241 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8495498-a035-43d2-b014-b3a93e0e50cb None admin] RBAC: Adding query filter params (role.id=83cf0a194f8d45e792411e2d9afdfe50, scope.project.id=b707733ce65c42edbc1fd95ad0a1c06c, user.id=90492440f6a6471e8392361c098428aa) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:17:49.141726 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8495498-a035-43d2-b014-b3a93e0e50cb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:17:49.144000 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8495498-a035-43d2-b014-b3a93e0e50cb None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:17:49.153563 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 334/683] 10.222.0.30 () {56 vars in 1374 bytes} [Tue Sep 6 04:17:48 2022] GET /identity/v3/role_assignments?user.id=90492440f6a6471e8392361c098428aa&scope.project.id=b707733ce65c42edbc1fd95ad0a1c06c&role.id=83cf0a194f8d45e792411e2d9afdfe50 => generated 608 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:17:49.249743 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7100ccf7-dd42-4778-9eed-33a7b9f2c06d None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:49.250178 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7100ccf7-dd42-4778-9eed-33a7b9f2c06d None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:49.250653 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7100ccf7-dd42-4778-9eed-33a7b9f2c06d None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:49.251199 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a785bb3-d37d-480d-a7fa-113cceb8816e None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:49.252844 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 335/684] 10.222.0.30 () {54 vars in 927 bytes} [Tue Sep 6 04:17:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 06 04:17:49.259512 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab2e375-44ce-4d55-a635-a1808189c9cf None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:49.259912 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab2e375-44ce-4d55-a635-a1808189c9cf None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:49.260297 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab2e375-44ce-4d55-a635-a1808189c9cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:49.280931 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:17:49.293469 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-2ab2e375-44ce-4d55-a635-a1808189c9cf None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:17:49.294709 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e06296d-b672-436c-8d9f-5954410454c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:49.294942 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-2ab2e375-44ce-4d55-a635-a1808189c9cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:17:49.295645 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e06296d-b672-436c-8d9f-5954410454c2 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:49.296321 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e06296d-b672-436c-8d9f-5954410454c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:49.296736 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e06296d-b672-436c-8d9f-5954410454c2 None admin] PATH_INFO: `/v3/users` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:49.297683 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e06296d-b672-436c-8d9f-5954410454c2 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:17:49.298216 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e06296d-b672-436c-8d9f-5954410454c2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:17:49.299862 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e06296d-b672-436c-8d9f-5954410454c2 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:17:49.322308 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 350/685] 10.222.0.30 () {56 vars in 1135 bytes} [Tue Sep 6 04:17:49 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:17:49.324001 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdc3baf-1535-4077-9930-4e13c41db687 None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:49.324433 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdc3baf-1535-4077-9930-4e13c41db687 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:49.324848 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdc3baf-1535-4077-9930-4e13c41db687 None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:49.326693 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 351/686] 10.222.0.30 () {54 vars in 927 bytes} [Tue Sep 6 04:17:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 04:17:49.327661 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccdc3baf-1535-4077-9930-4e13c41db687 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:49.341943 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:17:49.401820 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 336/687] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Sep 6 04:17:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 04:17:49.403005 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd60e14d-f282-48c7-98ad-546f7d3b6eb4 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:49.403298 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd60e14d-f282-48c7-98ad-546f7d3b6eb4 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:49.403582 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd60e14d-f282-48c7-98ad-546f7d3b6eb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:49.442485 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-cd60e14d-f282-48c7-98ad-546f7d3b6eb4 None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:17:49.443708 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-cd60e14d-f282-48c7-98ad-546f7d3b6eb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:17:49.460629 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccdc3baf-1535-4077-9930-4e13c41db687 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:49.507881 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05301bab-4c02-4da3-9f08-137831910a43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:49.508898 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05301bab-4c02-4da3-9f08-137831910a43 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:49.509426 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05301bab-4c02-4da3-9f08-137831910a43 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:49.509868 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05301bab-4c02-4da3-9f08-137831910a43 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:49.549669 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 337/688] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Sep 6 04:17:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 04:17:49.586476 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:17:49.616658 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05301bab-4c02-4da3-9f08-137831910a43 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:17:49.618160 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05301bab-4c02-4da3-9f08-137831910a43 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:17:49.743801 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 352/689] 10.222.0.30 () {58 vars in 1282 bytes} [Tue Sep 6 04:17:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 416 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 06 04:17:49.748957 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd60e14d-f282-48c7-98ad-546f7d3b6eb4 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:49.827144 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05301bab-4c02-4da3-9f08-137831910a43 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:49.877961 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd60e14d-f282-48c7-98ad-546f7d3b6eb4 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:49.893526 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:17:49.931764 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e14f9a-6853-4ad6-b257-a13ca007de05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:49.932471 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e14f9a-6853-4ad6-b257-a13ca007de05 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:49.932755 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e14f9a-6853-4ad6-b257-a13ca007de05 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:49.933028 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e14f9a-6853-4ad6-b257-a13ca007de05 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:49.956076 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e14f9a-6853-4ad6-b257-a13ca007de05 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:17:49.958195 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e14f9a-6853-4ad6-b257-a13ca007de05 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:17:49.987726 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-a0e14f9a-6853-4ad6-b257-a13ca007de05 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 04:17:49.987726 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 04:17:49.987726 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 04:17:49.987726 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 04:17:49.987726 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 04:17:49.987726 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 04:17:49.987726 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 04:17:49.987726 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 04:17:49.987726 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 04:17:49.987726 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 04:17:49.987726 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 04:17:49.987726 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 04:17:49.987726 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 04:17:49.987726 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 04:17:49.987726 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 04:17:49.987726 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 04:17:49.988953 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 04:17:49.988953 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 04:17:49.988953 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 04:17:49.988953 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 04:17:49.988953 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 04:17:49.988953 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 04:17:49.988953 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 04:17:49.988953 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 04:17:49.988953 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 04:17:49.988953 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 04:17:49.988953 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 04:17:49.988953 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 04:17:49.988953 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 04:17:49.988953 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 04:17:49.988953 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 04:17:49.988953 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 04:17:49.988953 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 04:17:49.989897 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 04:17:49.989897 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 04:17:49.989897 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 04:17:49.989897 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 338/690] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Sep 6 04:17:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 04:17:49.994254 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e14f9a-6853-4ad6-b257-a13ca007de05 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:50.007415 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:17:50.016940 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05301bab-4c02-4da3-9f08-137831910a43 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:50.060159 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23367bfa-0007-4e47-8750-4139c24b72fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:50.060815 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23367bfa-0007-4e47-8750-4139c24b72fc None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:50.061054 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23367bfa-0007-4e47-8750-4139c24b72fc None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:50.061346 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23367bfa-0007-4e47-8750-4139c24b72fc None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:50.074587 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23367bfa-0007-4e47-8750-4139c24b72fc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 04:17:50.081818 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-23367bfa-0007-4e47-8750-4139c24b72fc None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 04:17:50.081818 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 04:17:50.081818 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 04:17:50.081818 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 04:17:50.081818 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 04:17:50.081818 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 04:17:50.081818 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 04:17:50.081818 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 04:17:50.081818 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 04:17:50.081818 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 04:17:50.081818 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 04:17:50.081818 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 04:17:50.081818 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 04:17:50.081818 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 04:17:50.081818 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 04:17:50.081818 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 04:17:50.083130 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 04:17:50.083130 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 04:17:50.083130 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 04:17:50.083130 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 04:17:50.083130 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 04:17:50.083130 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 04:17:50.083130 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 04:17:50.083130 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 04:17:50.083130 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 04:17:50.083130 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 04:17:50.083130 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 04:17:50.083130 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 04:17:50.083130 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 04:17:50.083130 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 04:17:50.083130 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 04:17:50.083130 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 04:17:50.083988 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 04:17:50.083988 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 04:17:50.083988 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 04:17:50.083988 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 04:17:50.083988 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 04:17:50.083988 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 04:17:50.083988 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 04:17:50.083988 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 04:17:50.083988 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 04:17:50.083988 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 04:17:50.083988 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 04:17:50.083988 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 04:17:50.083988 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 353/691] 10.222.0.30 () {56 vars in 1096 bytes} [Tue Sep 6 04:17:49 2022] GET /identity/v3/roles/admin => generated 83 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 04:17:50.084692 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23367bfa-0007-4e47-8750-4139c24b72fc None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:50.118511 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e14f9a-6853-4ad6-b257-a13ca007de05 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:50.161858 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5990f310-3044-4734-9af1-2c97bb090bc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:50.162591 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5990f310-3044-4734-9af1-2c97bb090bc5 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:50.162881 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5990f310-3044-4734-9af1-2c97bb090bc5 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:50.163163 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5990f310-3044-4734-9af1-2c97bb090bc5 None admin] PATH_INFO: `/v3/domains` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:50.163818 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5990f310-3044-4734-9af1-2c97bb090bc5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:17:50.164127 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5990f310-3044-4734-9af1-2c97bb090bc5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:17:50.165624 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5990f310-3044-4734-9af1-2c97bb090bc5 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:17:50.183822 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 339/692] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Sep 6 04:17:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:17:50.184735 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5990f310-3044-4734-9af1-2c97bb090bc5 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:50.197795 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:17:50.201903 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23367bfa-0007-4e47-8750-4139c24b72fc None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:50.238806 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd3e27c-fda2-48c2-9818-9b6352aed012 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:50.239289 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd3e27c-fda2-48c2-9818-9b6352aed012 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:50.239569 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd3e27c-fda2-48c2-9818-9b6352aed012 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:50.239843 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd3e27c-fda2-48c2-9818-9b6352aed012 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:50.251166 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd3e27c-fda2-48c2-9818-9b6352aed012 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:17:50.252825 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd3e27c-fda2-48c2-9818-9b6352aed012 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:17:50.255031 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:17:50.269842 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-9bd3e27c-fda2-48c2-9818-9b6352aed012 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 04:17:50.269842 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 04:17:50.269842 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 04:17:50.269842 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 04:17:50.269842 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 04:17:50.269842 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 04:17:50.269842 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 04:17:50.269842 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 04:17:50.269842 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 04:17:50.269842 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 04:17:50.269842 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 04:17:50.269842 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 04:17:50.269842 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 04:17:50.269842 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 04:17:50.269842 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 04:17:50.269842 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 04:17:50.271039 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 04:17:50.271039 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 04:17:50.271039 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 04:17:50.271039 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 04:17:50.271039 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 04:17:50.271039 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 04:17:50.271039 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 04:17:50.271039 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 04:17:50.271039 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 04:17:50.271039 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 04:17:50.271039 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 04:17:50.271039 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 04:17:50.271039 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 04:17:50.271039 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 04:17:50.271039 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 04:17:50.271039 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 04:17:50.271039 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 04:17:50.272149 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 04:17:50.272149 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 04:17:50.272149 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 04:17:50.275860 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 354/693] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Sep 6 04:17:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 04:17:50.276819 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd3e27c-fda2-48c2-9818-9b6352aed012 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:50.303470 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5990f310-3044-4734-9af1-2c97bb090bc5 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:50.344632 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5dafaf-0e26-4dfb-8dcd-a3bcd59cb4a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:50.345249 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5dafaf-0e26-4dfb-8dcd-a3bcd59cb4a9 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:50.346302 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5dafaf-0e26-4dfb-8dcd-a3bcd59cb4a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:50.346567 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5dafaf-0e26-4dfb-8dcd-a3bcd59cb4a9 None admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:50.347317 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5dafaf-0e26-4dfb-8dcd-a3bcd59cb4a9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:17:50.347618 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5dafaf-0e26-4dfb-8dcd-a3bcd59cb4a9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:17:50.349109 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5dafaf-0e26-4dfb-8dcd-a3bcd59cb4a9 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:17:50.361563 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 340/694] 10.222.0.30 () {56 vars in 1099 bytes} [Tue Sep 6 04:17:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:17:50.366110 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5dafaf-0e26-4dfb-8dcd-a3bcd59cb4a9 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:50.394084 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd3e27c-fda2-48c2-9818-9b6352aed012 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:50.433773 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b03fe5d-b1b4-48ca-a56e-818206815e29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:50.434793 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b03fe5d-b1b4-48ca-a56e-818206815e29 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:50.434793 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b03fe5d-b1b4-48ca-a56e-818206815e29 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:50.435062 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b03fe5d-b1b4-48ca-a56e-818206815e29 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:50.444248 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b03fe5d-b1b4-48ca-a56e-818206815e29 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:17:50.446073 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b03fe5d-b1b4-48ca-a56e-818206815e29 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:17:50.451132 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:17:50.459683 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-3b03fe5d-b1b4-48ca-a56e-818206815e29 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 04:17:50.459683 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 04:17:50.459683 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 04:17:50.459683 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 04:17:50.459683 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 04:17:50.459683 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 04:17:50.459683 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 04:17:50.459683 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 04:17:50.459683 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 04:17:50.459683 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 04:17:50.459683 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 04:17:50.459683 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 04:17:50.459683 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 06 04:17:50.459683 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 06 04:17:50.459683 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 06 04:17:50.460778 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 06 04:17:50.460778 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 04:17:50.460778 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 04:17:50.460778 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 04:17:50.460778 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 04:17:50.460778 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 04:17:50.460778 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 04:17:50.460778 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 04:17:50.460778 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 04:17:50.460778 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 04:17:50.460778 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 04:17:50.460778 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 04:17:50.460778 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 04:17:50.460778 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 04:17:50.460778 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 04:17:50.460778 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 04:17:50.460778 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 04:17:50.461779 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 06 04:17:50.461779 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 06 04:17:50.461779 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 06 04:17:50.461779 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 06 04:17:50.461779 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 06 04:17:50.461779 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 06 04:17:50.461779 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 04:17:50.461779 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 04:17:50.463767 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 355/695] 10.222.0.30 () {56 vars in 1111 bytes} [Tue Sep 6 04:17:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 04:17:50.464682 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b03fe5d-b1b4-48ca-a56e-818206815e29 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:50.488013 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5dafaf-0e26-4dfb-8dcd-a3bcd59cb4a9 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:50.527130 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa2f0f3-031d-44cd-80c9-011a00e0a5b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:50.527765 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa2f0f3-031d-44cd-80c9-011a00e0a5b6 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:50.527951 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa2f0f3-031d-44cd-80c9-011a00e0a5b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:50.528225 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa2f0f3-031d-44cd-80c9-011a00e0a5b6 None admin] PATH_INFO: `/v3/domains` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:50.528874 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa2f0f3-031d-44cd-80c9-011a00e0a5b6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:17:50.529262 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa2f0f3-031d-44cd-80c9-011a00e0a5b6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:17:50.530627 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa2f0f3-031d-44cd-80c9-011a00e0a5b6 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:17:50.538614 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:17:50.548863 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 341/696] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Sep 6 04:17:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:17:50.548863 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa2f0f3-031d-44cd-80c9-011a00e0a5b6 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:50.593635 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b03fe5d-b1b4-48ca-a56e-818206815e29 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:50.632139 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd41191-7dd2-4ead-9617-d9c3a47687cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:50.633013 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd41191-7dd2-4ead-9617-d9c3a47687cd None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:50.633585 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd41191-7dd2-4ead-9617-d9c3a47687cd None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:50.634004 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd41191-7dd2-4ead-9617-d9c3a47687cd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:50.645787 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd41191-7dd2-4ead-9617-d9c3a47687cd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:17:50.647626 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd41191-7dd2-4ead-9617-d9c3a47687cd None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:17:50.655826 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa2f0f3-031d-44cd-80c9-011a00e0a5b6 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:50.663506 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-1dd41191-7dd2-4ead-9617-d9c3a47687cd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 04:17:50.663506 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 04:17:50.663506 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 04:17:50.663506 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 04:17:50.663506 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 04:17:50.663506 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 04:17:50.663506 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 04:17:50.663506 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 04:17:50.663506 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 04:17:50.663506 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 04:17:50.663506 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 04:17:50.663506 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 04:17:50.663506 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 04:17:50.663506 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 04:17:50.663506 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 04:17:50.663506 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 04:17:50.664607 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 04:17:50.664607 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 04:17:50.664607 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 04:17:50.664607 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 04:17:50.664607 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 04:17:50.664607 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 04:17:50.664607 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 04:17:50.664607 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 04:17:50.664607 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 04:17:50.664607 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 04:17:50.664607 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 04:17:50.664607 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 04:17:50.664607 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 04:17:50.664607 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 04:17:50.664607 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 04:17:50.664607 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 04:17:50.664607 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 04:17:50.665669 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 04:17:50.665669 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 04:17:50.665669 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 04:17:50.667568 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 356/697] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Sep 6 04:17:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 04:17:50.668458 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd41191-7dd2-4ead-9617-d9c3a47687cd None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:50.692782 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:17:50.698249 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8549beb6-c5d6-4693-869c-ee1a4058acfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:50.698783 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8549beb6-c5d6-4693-869c-ee1a4058acfc None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:50.699068 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8549beb6-c5d6-4693-869c-ee1a4058acfc None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:50.699346 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8549beb6-c5d6-4693-869c-ee1a4058acfc None admin] PATH_INFO: `/v3/projects` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:50.700046 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8549beb6-c5d6-4693-869c-ee1a4058acfc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:17:50.700361 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8549beb6-c5d6-4693-869c-ee1a4058acfc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:17:50.701966 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8549beb6-c5d6-4693-869c-ee1a4058acfc None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:17:50.719799 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 342/698] 10.222.0.30 () {56 vars in 1148 bytes} [Tue Sep 6 04:17:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:17:50.720830 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8549beb6-c5d6-4693-869c-ee1a4058acfc None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:50.782208 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd41191-7dd2-4ead-9617-d9c3a47687cd None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:50.790691 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:17:50.820914 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ca8b07-8cad-41fa-8076-aa7fea63f420 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:50.821850 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ca8b07-8cad-41fa-8076-aa7fea63f420 None admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:50.822375 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ca8b07-8cad-41fa-8076-aa7fea63f420 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:50.822795 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ca8b07-8cad-41fa-8076-aa7fea63f420 None admin] PATH_INFO: `/v3/users` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:50.823618 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ca8b07-8cad-41fa-8076-aa7fea63f420 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:17:50.825349 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ca8b07-8cad-41fa-8076-aa7fea63f420 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:17:50.828412 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8549beb6-c5d6-4693-869c-ee1a4058acfc None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:50.865228 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9cfac92-06c6-4456-8d12-b710666c6ee0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:50.865869 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cfac92-06c6-4456-8d12-b710666c6ee0 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:50.866152 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cfac92-06c6-4456-8d12-b710666c6ee0 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:50.866427 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cfac92-06c6-4456-8d12-b710666c6ee0 None admin] PATH_INFO: `/v3/domains` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:50.867079 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9cfac92-06c6-4456-8d12-b710666c6ee0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:17:50.867387 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9cfac92-06c6-4456-8d12-b710666c6ee0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:17:50.868769 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9cfac92-06c6-4456-8d12-b710666c6ee0 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:17:50.887523 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 343/699] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Sep 6 04:17:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:17:50.887523 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9cfac92-06c6-4456-8d12-b710666c6ee0 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:50.916673 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 357/700] 10.222.0.30 () {60 vars in 1132 bytes} [Tue Sep 6 04:17:50 2022] POST /identity/v3/users => generated 254 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 04:17:50.916673 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ca8b07-8cad-41fa-8076-aa7fea63f420 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:50.983563 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:17:51.097847 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9cfac92-06c6-4456-8d12-b710666c6ee0 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:51.126729 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ca8b07-8cad-41fa-8076-aa7fea63f420 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:51.141915 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1748c7-614b-4e61-ad41-5d83a3a4bdd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:51.142946 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1748c7-614b-4e61-ad41-5d83a3a4bdd4 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:51.143425 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1748c7-614b-4e61-ad41-5d83a3a4bdd4 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:51.143956 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1748c7-614b-4e61-ad41-5d83a3a4bdd4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:51.144957 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e1748c7-614b-4e61-ad41-5d83a3a4bdd4 None admin] RBAC: Adding query filter params (role.id=83cf0a194f8d45e792411e2d9afdfe50, scope.project.id=b707733ce65c42edbc1fd95ad0a1c06c, user.id=a676ec7f92a144edbeca8e6029ba9ee6) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:17:51.145498 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e1748c7-614b-4e61-ad41-5d83a3a4bdd4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:17:51.147400 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e1748c7-614b-4e61-ad41-5d83a3a4bdd4 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:17:51.154619 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 344/701] 10.222.0.30 () {56 vars in 1374 bytes} [Tue Sep 6 04:17:50 2022] GET /identity/v3/role_assignments?user.id=a676ec7f92a144edbeca8e6029ba9ee6&scope.project.id=b707733ce65c42edbc1fd95ad0a1c06c&role.id=83cf0a194f8d45e792411e2d9afdfe50 => generated 608 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:17:51.170941 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e3919f-af96-4686-8733-a5f30a78aa9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:51.172052 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e3919f-af96-4686-8733-a5f30a78aa9e None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:51.172552 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e3919f-af96-4686-8733-a5f30a78aa9e None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:51.173043 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e3919f-af96-4686-8733-a5f30a78aa9e None admin] PATH_INFO: `/v3/users/nova` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:51.187342 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e3919f-af96-4686-8733-a5f30a78aa9e None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:17:51.189291 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e3919f-af96-4686-8733-a5f30a78aa9e None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:17:51.209896 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-22e3919f-af96-4686-8733-a5f30a78aa9e None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 06 04:17:51.209896 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 04:17:51.209896 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 04:17:51.209896 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 04:17:51.209896 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 04:17:51.209896 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 04:17:51.209896 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 04:17:51.209896 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 04:17:51.209896 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 04:17:51.209896 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 04:17:51.209896 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 04:17:51.209896 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 04:17:51.209896 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 06 04:17:51.209896 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 06 04:17:51.209896 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 06 04:17:51.209896 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 06 04:17:51.210897 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 04:17:51.210897 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 04:17:51.210897 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 06 04:17:51.210897 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 04:17:51.210897 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 06 04:17:51.210897 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 04:17:51.210897 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 04:17:51.210897 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 04:17:51.210897 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 04:17:51.210897 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 04:17:51.210897 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 04:17:51.210897 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 04:17:51.210897 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 04:17:51.210897 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 04:17:51.210897 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 04:17:51.210897 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 04:17:51.210897 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 04:17:51.210897 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 04:17:51.211912 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 04:17:51.211912 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 04:17:51.211912 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 06 04:17:51.211912 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 06 04:17:51.211912 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 06 04:17:51.211912 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 06 04:17:51.211912 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 06 04:17:51.211912 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 06 04:17:51.211912 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 06 04:17:51.211912 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 04:17:51.215356 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 358/702] 10.222.0.30 () {56 vars in 1093 bytes} [Tue Sep 6 04:17:50 2022] GET /identity/v3/users/nova => generated 82 bytes in 301 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 04:17:51.221589 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1748c7-614b-4e61-ad41-5d83a3a4bdd4 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:51.334877 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcf5e24-9678-4594-a60e-bf53a8cb0be2 None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:51.336029 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcf5e24-9678-4594-a60e-bf53a8cb0be2 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:51.336633 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcf5e24-9678-4594-a60e-bf53a8cb0be2 None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:51.338782 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 359/703] 10.222.0.30 () {54 vars in 927 bytes} [Tue Sep 6 04:17:51 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 06 04:17:51.346244 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6796102-ea13-4fc5-aaea-38959c636ce2 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:51.346529 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6796102-ea13-4fc5-aaea-38959c636ce2 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:51.346866 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6796102-ea13-4fc5-aaea-38959c636ce2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:51.358510 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1748c7-614b-4e61-ad41-5d83a3a4bdd4 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:51.368235 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-e6796102-ea13-4fc5-aaea-38959c636ce2 None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:17:51.369493 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-e6796102-ea13-4fc5-aaea-38959c636ce2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:17:51.376227 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:17:51.397966 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d7d673-1be3-4d70-9b4b-b5d80b9c7400 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:51.398715 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d7d673-1be3-4d70-9b4b-b5d80b9c7400 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:51.399101 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d7d673-1be3-4d70-9b4b-b5d80b9c7400 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:51.399480 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d7d673-1be3-4d70-9b4b-b5d80b9c7400 None admin] PATH_INFO: `/v3/users` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:51.400324 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d7d673-1be3-4d70-9b4b-b5d80b9c7400 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:17:51.400735 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d7d673-1be3-4d70-9b4b-b5d80b9c7400 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:17:51.402325 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d7d673-1be3-4d70-9b4b-b5d80b9c7400 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:17:51.403869 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:17:51.424373 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 345/704] 10.222.0.30 () {56 vars in 1133 bytes} [Tue Sep 6 04:17:51 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:17:51.426501 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338f70a2-b62d-4765-be10-4ac382ff85cd None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:51.427659 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338f70a2-b62d-4765-be10-4ac382ff85cd None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:51.428018 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338f70a2-b62d-4765-be10-4ac382ff85cd None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:51.429757 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 346/705] 10.222.0.30 () {54 vars in 927 bytes} [Tue Sep 6 04:17:51 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 04:17:51.430538 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338f70a2-b62d-4765-be10-4ac382ff85cd None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:51.473505 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 360/706] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Sep 6 04:17:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 06 04:17:51.474721 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d912b3-c744-4dd3-81d5-ca1e6e92c145 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:51.475112 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d912b3-c744-4dd3-81d5-ca1e6e92c145 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:51.475488 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d912b3-c744-4dd3-81d5-ca1e6e92c145 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:51.494435 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-85d912b3-c744-4dd3-81d5-ca1e6e92c145 None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:17:51.495764 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-85d912b3-c744-4dd3-81d5-ca1e6e92c145 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:17:51.618833 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 361/707] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Sep 6 04:17:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 04:17:51.645713 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338f70a2-b62d-4765-be10-4ac382ff85cd None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:51.683564 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957ed326-881d-46d0-8c83-50ac10bc01c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:51.684177 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957ed326-881d-46d0-8c83-50ac10bc01c5 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:51.684454 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957ed326-881d-46d0-8c83-50ac10bc01c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:51.684724 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957ed326-881d-46d0-8c83-50ac10bc01c5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:51.738631 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:17:51.767043 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957ed326-881d-46d0-8c83-50ac10bc01c5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:17:51.768484 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957ed326-881d-46d0-8c83-50ac10bc01c5 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:17:51.930094 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 347/708] 10.222.0.30 () {58 vars in 1282 bytes} [Tue Sep 6 04:17:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 500 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 06 04:17:51.931797 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d912b3-c744-4dd3-81d5-ca1e6e92c145 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:51.940438 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957ed326-881d-46d0-8c83-50ac10bc01c5 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:52.046787 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:17:52.054498 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:17:52.072975 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d912b3-c744-4dd3-81d5-ca1e6e92c145 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:52.086812 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957ed326-881d-46d0-8c83-50ac10bc01c5 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:52.110842 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba743b25-bc38-4ade-bcf2-985988e08e4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:52.111679 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba743b25-bc38-4ade-bcf2-985988e08e4f None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:52.111801 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba743b25-bc38-4ade-bcf2-985988e08e4f None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:52.112084 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba743b25-bc38-4ade-bcf2-985988e08e4f None admin] PATH_INFO: `/v3/services/network` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:52.112729 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba743b25-bc38-4ade-bcf2-985988e08e4f None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:17:52.115009 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba743b25-bc38-4ade-bcf2-985988e08e4f None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:17:52.130351 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-ba743b25-bc38-4ade-bcf2-985988e08e4f None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 06 04:17:52.130351 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 04:17:52.130351 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 06 04:17:52.130351 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 06 04:17:52.130351 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 06 04:17:52.130351 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 06 04:17:52.130351 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 06 04:17:52.130351 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 04:17:52.130351 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 06 04:17:52.130351 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 04:17:52.130351 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 06 04:17:52.130351 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 04:17:52.130351 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 04:17:52.130351 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 04:17:52.130351 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 04:17:52.130351 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 04:17:52.130351 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 04:17:52.136852 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 04:17:52.136852 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 04:17:52.136852 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 04:17:52.136852 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 04:17:52.136852 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 04:17:52.136852 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 04:17:52.136852 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 06 04:17:52.136852 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 06 04:17:52.136852 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 06 04:17:52.136852 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 06 04:17:52.136852 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 04:17:52.136852 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 04:17:52.136852 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 04:17:52.136852 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 04:17:52.136852 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 04:17:52.136852 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 04:17:52.138926 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 04:17:52.138926 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 04:17:52.138926 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 04:17:52.138926 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 04:17:52.138926 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 04:17:52.138926 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 04:17:52.138926 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 04:17:52.138926 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 04:17:52.138926 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 04:17:52.138926 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 04:17:52.138926 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 06 04:17:52.138926 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 04:17:52.138926 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 06 04:17:52.138926 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 04:17:52.138926 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 362/709] 10.222.0.30 () {56 vars in 1111 bytes} [Tue Sep 6 04:17:51 2022] GET /identity/v3/services/network => generated 88 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 04:17:52.139750 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba743b25-bc38-4ade-bcf2-985988e08e4f None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:52.144227 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9baecca-df47-457e-9dc7-196144475a36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:52.145781 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9baecca-df47-457e-9dc7-196144475a36 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:52.146294 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9baecca-df47-457e-9dc7-196144475a36 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:52.146769 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9baecca-df47-457e-9dc7-196144475a36 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:52.161015 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9baecca-df47-457e-9dc7-196144475a36 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:17:52.163272 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9baecca-df47-457e-9dc7-196144475a36 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:17:52.183908 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-e9baecca-df47-457e-9dc7-196144475a36 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 04:17:52.183908 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 04:17:52.183908 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 04:17:52.183908 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 04:17:52.183908 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 04:17:52.183908 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 04:17:52.183908 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 04:17:52.183908 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 04:17:52.183908 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 04:17:52.183908 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 04:17:52.183908 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 04:17:52.183908 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 04:17:52.183908 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 04:17:52.183908 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 04:17:52.183908 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 04:17:52.183908 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 04:17:52.184923 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 04:17:52.184923 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 04:17:52.184923 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 04:17:52.184923 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 04:17:52.184923 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 04:17:52.184923 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 04:17:52.184923 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 04:17:52.184923 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 04:17:52.184923 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 04:17:52.184923 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 04:17:52.184923 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 04:17:52.184923 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 04:17:52.184923 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 04:17:52.184923 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 04:17:52.184923 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 04:17:52.184923 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 04:17:52.184923 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 04:17:52.185915 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 04:17:52.185915 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 04:17:52.185915 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 04:17:52.189385 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 348/710] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Sep 6 04:17:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 04:17:52.190401 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9baecca-df47-457e-9dc7-196144475a36 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:52.258228 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba743b25-bc38-4ade-bcf2-985988e08e4f None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:52.294589 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c39a56-5cc7-4647-af5c-421e2bdaa23b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:52.295018 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c39a56-5cc7-4647-af5c-421e2bdaa23b None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:52.295303 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c39a56-5cc7-4647-af5c-421e2bdaa23b None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:52.295580 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c39a56-5cc7-4647-af5c-421e2bdaa23b None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:52.296176 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c39a56-5cc7-4647-af5c-421e2bdaa23b None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:17:52.297577 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c39a56-5cc7-4647-af5c-421e2bdaa23b None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:17:52.307629 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-b7c39a56-5cc7-4647-af5c-421e2bdaa23b None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 06 04:17:52.307629 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 04:17:52.307629 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 06 04:17:52.307629 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 06 04:17:52.307629 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 06 04:17:52.307629 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 06 04:17:52.307629 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 06 04:17:52.307629 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 04:17:52.307629 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 06 04:17:52.307629 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 04:17:52.307629 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 06 04:17:52.307629 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 04:17:52.307629 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 04:17:52.307629 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 04:17:52.307629 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 04:17:52.307629 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 04:17:52.307629 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 04:17:52.308670 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 04:17:52.308670 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 04:17:52.308670 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 04:17:52.308670 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 04:17:52.308670 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 04:17:52.308670 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 04:17:52.308670 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 06 04:17:52.308670 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 06 04:17:52.308670 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 06 04:17:52.308670 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 06 04:17:52.308670 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 04:17:52.308670 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 04:17:52.308670 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 04:17:52.308670 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 04:17:52.308670 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 04:17:52.308670 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 04:17:52.314328 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 04:17:52.314328 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 04:17:52.314328 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 04:17:52.314328 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 04:17:52.314328 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 04:17:52.314328 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 04:17:52.314328 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 04:17:52.314328 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 04:17:52.314328 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 04:17:52.314328 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 04:17:52.314328 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 06 04:17:52.314328 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 04:17:52.314328 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 06 04:17:52.314328 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 04:17:52.314328 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 363/711] 10.222.0.30 () {56 vars in 1129 bytes} [Tue Sep 6 04:17:52 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 04:17:52.315200 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c39a56-5cc7-4647-af5c-421e2bdaa23b None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:52.319265 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9baecca-df47-457e-9dc7-196144475a36 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:52.323042 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:17:52.347409 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:17:52.357552 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7ecd78-741b-400a-ae11-10e647b15d26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:52.358566 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7ecd78-741b-400a-ae11-10e647b15d26 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:52.359049 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7ecd78-741b-400a-ae11-10e647b15d26 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:52.359633 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7ecd78-741b-400a-ae11-10e647b15d26 None admin] PATH_INFO: `/v3/services` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:52.360766 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7ecd78-741b-400a-ae11-10e647b15d26 None admin] RBAC: Adding query filter params (name=network) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:17:52.367166 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7ecd78-741b-400a-ae11-10e647b15d26 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:17:52.368821 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7ecd78-741b-400a-ae11-10e647b15d26 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:17:52.378927 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 349/712] 10.222.0.30 () {56 vars in 1112 bytes} [Tue Sep 6 04:17:52 2022] GET /identity/v3/services?name=network => generated 124 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:17:52.382973 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7ecd78-741b-400a-ae11-10e647b15d26 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:52.418204 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c39a56-5cc7-4647-af5c-421e2bdaa23b None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:52.462826 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d9ecc8-052b-4320-a689-0a01110142f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:52.463374 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d9ecc8-052b-4320-a689-0a01110142f7 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:52.463656 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d9ecc8-052b-4320-a689-0a01110142f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:52.463931 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d9ecc8-052b-4320-a689-0a01110142f7 None admin] PATH_INFO: `/v3/domains` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:52.464590 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d9ecc8-052b-4320-a689-0a01110142f7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:17:52.464902 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d9ecc8-052b-4320-a689-0a01110142f7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:17:52.466420 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d9ecc8-052b-4320-a689-0a01110142f7 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:17:52.481468 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 364/713] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Sep 6 04:17:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:17:52.482095 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d9ecc8-052b-4320-a689-0a01110142f7 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:52.497588 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7ecd78-741b-400a-ae11-10e647b15d26 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:52.534692 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da9bd87-4f5d-45c0-ba4e-f3e2f9dc5854 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:52.536160 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da9bd87-4f5d-45c0-ba4e-f3e2f9dc5854 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:52.536901 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da9bd87-4f5d-45c0-ba4e-f3e2f9dc5854 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:52.537435 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da9bd87-4f5d-45c0-ba4e-f3e2f9dc5854 None admin] PATH_INFO: `/v3/services` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:52.538313 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da9bd87-4f5d-45c0-ba4e-f3e2f9dc5854 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:17:52.538829 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da9bd87-4f5d-45c0-ba4e-f3e2f9dc5854 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:17:52.540819 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da9bd87-4f5d-45c0-ba4e-f3e2f9dc5854 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:17:52.551573 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 350/714] 10.222.0.30 () {56 vars in 1124 bytes} [Tue Sep 6 04:17:52 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:17:52.552781 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da9bd87-4f5d-45c0-ba4e-f3e2f9dc5854 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:52.605720 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:17:52.610026 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d9ecc8-052b-4320-a689-0a01110142f7 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:52.613979 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:17:52.647105 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53dc345c-f955-4817-8ae9-34ab7189cf85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:52.647645 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53dc345c-f955-4817-8ae9-34ab7189cf85 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:52.647927 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53dc345c-f955-4817-8ae9-34ab7189cf85 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:52.648203 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53dc345c-f955-4817-8ae9-34ab7189cf85 None admin] PATH_INFO: `/v3/services` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:52.648913 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53dc345c-f955-4817-8ae9-34ab7189cf85 None admin] RBAC: Adding query filter params (type=network) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:17:52.649221 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53dc345c-f955-4817-8ae9-34ab7189cf85 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:17:52.665037 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53dc345c-f955-4817-8ae9-34ab7189cf85 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:17:52.665037 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 365/715] 10.222.0.30 () {56 vars in 1112 bytes} [Tue Sep 6 04:17:52 2022] GET /identity/v3/services?type=network => generated 124 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:17:52.665037 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53dc345c-f955-4817-8ae9-34ab7189cf85 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:52.670424 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da9bd87-4f5d-45c0-ba4e-f3e2f9dc5854 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:52.745014 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc718509-dcc2-42dc-a604-229ac0c981e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:52.746036 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc718509-dcc2-42dc-a604-229ac0c981e5 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:52.746537 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc718509-dcc2-42dc-a604-229ac0c981e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:52.747007 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc718509-dcc2-42dc-a604-229ac0c981e5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:52.761327 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc718509-dcc2-42dc-a604-229ac0c981e5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:17:52.763284 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc718509-dcc2-42dc-a604-229ac0c981e5 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:17:52.772676 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53dc345c-f955-4817-8ae9-34ab7189cf85 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:52.789142 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-dc718509-dcc2-42dc-a604-229ac0c981e5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 04:17:52.789142 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 04:17:52.789142 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 04:17:52.789142 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 04:17:52.789142 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 04:17:52.789142 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 04:17:52.789142 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 04:17:52.789142 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 04:17:52.789142 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 04:17:52.789142 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 04:17:52.789142 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 04:17:52.789142 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 04:17:52.789142 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 06 04:17:52.789142 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 06 04:17:52.789142 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 06 04:17:52.790105 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 06 04:17:52.790105 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 04:17:52.790105 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 04:17:52.790105 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 04:17:52.790105 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 04:17:52.790105 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 04:17:52.790105 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 04:17:52.790105 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 04:17:52.790105 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 04:17:52.790105 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 04:17:52.790105 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 04:17:52.790105 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 04:17:52.790105 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 04:17:52.790105 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 04:17:52.790105 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 04:17:52.790105 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 04:17:52.790105 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 04:17:52.791008 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 06 04:17:52.791008 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 06 04:17:52.791008 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 06 04:17:52.791008 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 06 04:17:52.791008 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 06 04:17:52.791008 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 06 04:17:52.791008 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 04:17:52.791008 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 04:17:52.797582 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 351/716] 10.222.0.30 () {56 vars in 1111 bytes} [Tue Sep 6 04:17:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 04:17:52.798872 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc718509-dcc2-42dc-a604-229ac0c981e5 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:52.822768 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92ef9dc-7738-442a-81c4-98e2db5f4e71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:52.823457 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92ef9dc-7738-442a-81c4-98e2db5f4e71 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:52.823738 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92ef9dc-7738-442a-81c4-98e2db5f4e71 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:52.824063 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92ef9dc-7738-442a-81c4-98e2db5f4e71 None admin] PATH_INFO: `/v3/services` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:52.824718 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92ef9dc-7738-442a-81c4-98e2db5f4e71 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:17:52.825062 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92ef9dc-7738-442a-81c4-98e2db5f4e71 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:17:52.826591 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92ef9dc-7738-442a-81c4-98e2db5f4e71 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:17:52.833706 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:17:52.837852 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 366/717] 10.222.0.30 () {56 vars in 1124 bytes} [Tue Sep 6 04:17:52 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:17:52.956426 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc718509-dcc2-42dc-a604-229ac0c981e5 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:52.973449 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:17:53.005469 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d026e9-3d58-4c4e-a4ec-0892f0bcf02e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:53.006273 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d026e9-3d58-4c4e-a4ec-0892f0bcf02e None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:53.013387 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d026e9-3d58-4c4e-a4ec-0892f0bcf02e None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:53.013387 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d026e9-3d58-4c4e-a4ec-0892f0bcf02e None admin] PATH_INFO: `/v3/projects` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:53.013387 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d026e9-3d58-4c4e-a4ec-0892f0bcf02e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:17:53.013387 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d026e9-3d58-4c4e-a4ec-0892f0bcf02e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:17:53.013387 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d026e9-3d58-4c4e-a4ec-0892f0bcf02e None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:17:53.026840 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 352/718] 10.222.0.30 () {56 vars in 1148 bytes} [Tue Sep 6 04:17:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:17:53.032546 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92ef9dc-7738-442a-81c4-98e2db5f4e71 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:53.150552 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92ef9dc-7738-442a-81c4-98e2db5f4e71 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:53.198930 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf0e344-b1ea-45c7-a1db-622653d38b1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:53.199553 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf0e344-b1ea-45c7-a1db-622653d38b1c None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:53.199837 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf0e344-b1ea-45c7-a1db-622653d38b1c None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:53.200115 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf0e344-b1ea-45c7-a1db-622653d38b1c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:53.200845 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf0e344-b1ea-45c7-a1db-622653d38b1c None admin] RBAC: Adding query filter params (role.id=ea4c814713b149108e488de855b025e4, scope.project.id=b707733ce65c42edbc1fd95ad0a1c06c, user.id=9db138b77afa4b068244b8ef715404fe) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:17:53.201156 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf0e344-b1ea-45c7-a1db-622653d38b1c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:17:53.202623 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf0e344-b1ea-45c7-a1db-622653d38b1c None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:17:53.213732 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 367/719] 10.222.0.30 () {56 vars in 1374 bytes} [Tue Sep 6 04:17:53 2022] GET /identity/v3/role_assignments?user.id=9db138b77afa4b068244b8ef715404fe&scope.project.id=b707733ce65c42edbc1fd95ad0a1c06c&role.id=ea4c814713b149108e488de855b025e4 => generated 259 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:17:53.325152 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb767fab-73d5-499d-8533-86a6fe77513d None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:53.325506 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb767fab-73d5-499d-8533-86a6fe77513d None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:53.325758 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb767fab-73d5-499d-8533-86a6fe77513d None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:53.327632 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 353/720] 10.222.0.30 () {54 vars in 927 bytes} [Tue Sep 6 04:17:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 04:17:53.334330 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53718f8c-9b55-477d-863f-8ff8e62e462d None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:53.334646 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53718f8c-9b55-477d-863f-8ff8e62e462d None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:53.334895 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53718f8c-9b55-477d-863f-8ff8e62e462d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:53.363054 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-53718f8c-9b55-477d-863f-8ff8e62e462d None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:17:53.364508 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-53718f8c-9b55-477d-863f-8ff8e62e462d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:17:53.418426 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2316f8-c69f-43b5-bb42-5b48e44c6858 None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:53.418756 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2316f8-c69f-43b5-bb42-5b48e44c6858 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:53.418996 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2316f8-c69f-43b5-bb42-5b48e44c6858 None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:53.420553 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 354/721] 10.222.0.30 () {54 vars in 927 bytes} [Tue Sep 6 04:17:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 04:17:53.426843 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e13eee-92f5-4ef0-ad86-024b01f6d751 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:53.427118 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e13eee-92f5-4ef0-ad86-024b01f6d751 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:53.427398 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e13eee-92f5-4ef0-ad86-024b01f6d751 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:53.449693 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:17:53.451052 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-c0e13eee-92f5-4ef0-ad86-024b01f6d751 None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:17:53.452469 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-c0e13eee-92f5-4ef0-ad86-024b01f6d751 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:17:53.466392 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 368/722] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Sep 6 04:17:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 04:17:53.528090 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 355/723] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Sep 6 04:17:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 06 04:17:53.795398 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53718f8c-9b55-477d-863f-8ff8e62e462d None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:53.907551 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53718f8c-9b55-477d-863f-8ff8e62e462d None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:53.928488 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e13eee-92f5-4ef0-ad86-024b01f6d751 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:53.953835 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9d308b-48bc-42f1-89ca-4523fca28d92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:53.954483 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9d308b-48bc-42f1-89ca-4523fca28d92 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:53.954765 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9d308b-48bc-42f1-89ca-4523fca28d92 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:53.955037 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9d308b-48bc-42f1-89ca-4523fca28d92 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:53.985950 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9d308b-48bc-42f1-89ca-4523fca28d92 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 04:17:53.991493 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-8a9d308b-48bc-42f1-89ca-4523fca28d92 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 06 04:17:53.991493 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 04:17:53.991493 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 04:17:53.991493 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 04:17:53.991493 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 04:17:53.991493 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 04:17:53.991493 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 04:17:53.991493 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 04:17:53.991493 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 04:17:53.991493 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 04:17:53.991493 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 04:17:53.991493 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 04:17:53.991493 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 04:17:53.991493 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 04:17:53.991493 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 04:17:53.991493 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 04:17:53.992510 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 04:17:53.992510 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 04:17:53.992510 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 04:17:53.992510 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 04:17:53.992510 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 04:17:53.992510 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 04:17:53.992510 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 04:17:53.992510 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 04:17:53.992510 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 04:17:53.992510 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 04:17:53.992510 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 04:17:53.992510 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 04:17:53.992510 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 04:17:53.992510 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 04:17:53.992510 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 04:17:53.992510 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 04:17:53.993448 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 04:17:53.993448 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 04:17:53.993448 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 04:17:53.993448 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 04:17:53.993448 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 04:17:53.993448 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 04:17:53.993448 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 04:17:53.993448 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 04:17:53.993448 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 04:17:53.993448 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 04:17:53.993448 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 06 04:17:53.993448 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 04:17:53.997024 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 369/724] 10.222.0.30 () {56 vars in 1102 bytes} [Tue Sep 6 04:17:53 2022] GET /identity/v3/roles/service => generated 85 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 04:17:54.004644 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9d308b-48bc-42f1-89ca-4523fca28d92 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:54.103792 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:17:54.112577 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e13eee-92f5-4ef0-ad86-024b01f6d751 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:54.129689 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:17:54.157803 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9d308b-48bc-42f1-89ca-4523fca28d92 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:54.182699 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d692cc51-a7fc-4368-9004-3487e6bda5c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:54.183449 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d692cc51-a7fc-4368-9004-3487e6bda5c2 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:54.183735 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d692cc51-a7fc-4368-9004-3487e6bda5c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:54.184013 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d692cc51-a7fc-4368-9004-3487e6bda5c2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:54.196181 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d692cc51-a7fc-4368-9004-3487e6bda5c2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 04:17:54.196600 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7088ac3a-90f5-4865-8771-a22a3e467cae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:54.197477 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7088ac3a-90f5-4865-8771-a22a3e467cae None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:54.197974 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7088ac3a-90f5-4865-8771-a22a3e467cae None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:54.198542 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7088ac3a-90f5-4865-8771-a22a3e467cae None admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:54.199476 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7088ac3a-90f5-4865-8771-a22a3e467cae None admin] RBAC: Adding query filter params (name=service) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:17:54.200029 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7088ac3a-90f5-4865-8771-a22a3e467cae None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:17:54.200954 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-d692cc51-a7fc-4368-9004-3487e6bda5c2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 04:17:54.200954 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 04:17:54.200954 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 04:17:54.200954 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 04:17:54.200954 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 04:17:54.200954 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 04:17:54.200954 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 04:17:54.200954 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 04:17:54.200954 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 04:17:54.200954 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 04:17:54.200954 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 04:17:54.200954 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 04:17:54.200954 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 04:17:54.200954 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 04:17:54.200954 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 04:17:54.200954 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 04:17:54.201911 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 04:17:54.201911 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 04:17:54.201911 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 04:17:54.201911 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 04:17:54.201911 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 04:17:54.201911 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 04:17:54.201911 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 04:17:54.201911 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 04:17:54.201911 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 04:17:54.201911 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 04:17:54.201911 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 04:17:54.201911 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 04:17:54.201911 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 04:17:54.201911 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 04:17:54.201911 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 04:17:54.201911 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 04:17:54.202768 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 04:17:54.202768 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 04:17:54.202768 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 04:17:54.202768 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 04:17:54.202768 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 04:17:54.202768 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 04:17:54.202768 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 04:17:54.202768 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 04:17:54.202768 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 04:17:54.202768 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 04:17:54.202768 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 04:17:54.202768 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 04:17:54.204625 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7088ac3a-90f5-4865-8771-a22a3e467cae None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:17:54.206861 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 356/725] 10.222.0.30 () {56 vars in 1096 bytes} [Tue Sep 6 04:17:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 274 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 04:17:54.208973 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d692cc51-a7fc-4368-9004-3487e6bda5c2 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:54.215186 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 370/726] 10.222.0.30 () {56 vars in 1103 bytes} [Tue Sep 6 04:17:54 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:17:54.226820 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7088ac3a-90f5-4865-8771-a22a3e467cae None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:54.335389 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7088ac3a-90f5-4865-8771-a22a3e467cae None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:54.354160 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:17:54.372748 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944bf3c1-ca40-434d-aa00-554244508480 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:54.373827 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944bf3c1-ca40-434d-aa00-554244508480 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:54.374350 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944bf3c1-ca40-434d-aa00-554244508480 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:54.374892 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944bf3c1-ca40-434d-aa00-554244508480 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:54.387033 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d692cc51-a7fc-4368-9004-3487e6bda5c2 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:54.397279 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944bf3c1-ca40-434d-aa00-554244508480 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:17:54.399138 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944bf3c1-ca40-434d-aa00-554244508480 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:17:54.426228 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6ee71d-723c-4c38-82c3-f747f79ecb22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:54.427552 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6ee71d-723c-4c38-82c3-f747f79ecb22 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:54.427552 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6ee71d-723c-4c38-82c3-f747f79ecb22 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:54.427552 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6ee71d-723c-4c38-82c3-f747f79ecb22 None admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:54.428227 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6ee71d-723c-4c38-82c3-f747f79ecb22 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:17:54.428541 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6ee71d-723c-4c38-82c3-f747f79ecb22 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:17:54.430105 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6ee71d-723c-4c38-82c3-f747f79ecb22 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:17:54.431274 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-944bf3c1-ca40-434d-aa00-554244508480 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 04:17:54.431274 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 04:17:54.431274 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 04:17:54.431274 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 04:17:54.431274 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 04:17:54.431274 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 04:17:54.431274 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 04:17:54.431274 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 04:17:54.431274 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 04:17:54.431274 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 04:17:54.431274 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 04:17:54.431274 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 04:17:54.431274 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 04:17:54.431274 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 04:17:54.431274 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 04:17:54.431274 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 04:17:54.432244 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 04:17:54.432244 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 04:17:54.432244 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 04:17:54.432244 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 04:17:54.432244 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 04:17:54.432244 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 04:17:54.432244 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 04:17:54.432244 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 04:17:54.432244 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 04:17:54.432244 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 04:17:54.432244 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 04:17:54.432244 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 04:17:54.432244 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 04:17:54.432244 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 04:17:54.432244 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 04:17:54.432244 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 04:17:54.432244 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 04:17:54.433145 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 04:17:54.433145 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 04:17:54.433145 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 04:17:54.439553 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 371/728] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Sep 6 04:17:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 04:17:54.444435 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 357/728] 10.222.0.30 () {56 vars in 1099 bytes} [Tue Sep 6 04:17:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:17:54.448932 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944bf3c1-ca40-434d-aa00-554244508480 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:54.449076 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6ee71d-723c-4c38-82c3-f747f79ecb22 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:54.479167 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:17:54.634440 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6ee71d-723c-4c38-82c3-f747f79ecb22 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:54.670297 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b77330-50d7-4e41-a126-b6b5100353e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:54.671138 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b77330-50d7-4e41-a126-b6b5100353e5 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:54.671485 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b77330-50d7-4e41-a126-b6b5100353e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:54.671832 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b77330-50d7-4e41-a126-b6b5100353e5 None admin] PATH_INFO: `/v3/domains` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:54.672569 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b77330-50d7-4e41-a126-b6b5100353e5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:17:54.672942 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b77330-50d7-4e41-a126-b6b5100353e5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:17:54.674536 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b77330-50d7-4e41-a126-b6b5100353e5 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:17:54.689439 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944bf3c1-ca40-434d-aa00-554244508480 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:54.693481 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 358/729] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Sep 6 04:17:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:17:54.698031 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b77330-50d7-4e41-a126-b6b5100353e5 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:54.754314 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77f2ada-89d5-4697-a467-88cbd4684370 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:54.754978 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77f2ada-89d5-4697-a467-88cbd4684370 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:54.755266 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77f2ada-89d5-4697-a467-88cbd4684370 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:54.755546 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77f2ada-89d5-4697-a467-88cbd4684370 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:54.779370 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e77f2ada-89d5-4697-a467-88cbd4684370 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:17:54.780842 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e77f2ada-89d5-4697-a467-88cbd4684370 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:17:54.803628 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:17:54.813343 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-e77f2ada-89d5-4697-a467-88cbd4684370 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 04:17:54.813343 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 04:17:54.813343 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 04:17:54.813343 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 04:17:54.813343 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 04:17:54.813343 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 04:17:54.813343 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 04:17:54.813343 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 04:17:54.813343 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 04:17:54.813343 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 04:17:54.813343 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 04:17:54.813343 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 04:17:54.813343 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 04:17:54.813343 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 04:17:54.813343 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 04:17:54.813343 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 04:17:54.816118 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 04:17:54.816118 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 04:17:54.816118 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 04:17:54.816118 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 04:17:54.816118 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 04:17:54.816118 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 04:17:54.816118 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 04:17:54.816118 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 04:17:54.816118 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 04:17:54.816118 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 04:17:54.816118 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 04:17:54.816118 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 04:17:54.816118 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 04:17:54.816118 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 04:17:54.816118 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 04:17:54.816118 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 04:17:54.816118 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 04:17:54.818324 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 04:17:54.818324 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 04:17:54.818324 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 04:17:54.818324 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 372/730] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Sep 6 04:17:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 366 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 04:17:54.825329 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b77330-50d7-4e41-a126-b6b5100353e5 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:54.834339 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77f2ada-89d5-4697-a467-88cbd4684370 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:54.863348 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb5a3ba-bfd9-4384-b06d-00c4e4975570 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:54.863995 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb5a3ba-bfd9-4384-b06d-00c4e4975570 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:54.864274 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb5a3ba-bfd9-4384-b06d-00c4e4975570 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:54.866100 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb5a3ba-bfd9-4384-b06d-00c4e4975570 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:54.880644 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:17:54.885532 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb5a3ba-bfd9-4384-b06d-00c4e4975570 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:17:54.888145 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb5a3ba-bfd9-4384-b06d-00c4e4975570 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:17:54.916464 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-9bb5a3ba-bfd9-4384-b06d-00c4e4975570 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 06 04:17:54.916464 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 04:17:54.916464 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 04:17:54.916464 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 04:17:54.916464 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 04:17:54.916464 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 04:17:54.916464 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 04:17:54.916464 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 04:17:54.916464 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 04:17:54.916464 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 04:17:54.916464 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 04:17:54.916464 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 04:17:54.916464 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 06 04:17:54.916464 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 06 04:17:54.916464 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 06 04:17:54.916464 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 06 04:17:54.918773 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 04:17:54.918773 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 04:17:54.918773 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 06 04:17:54.918773 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 04:17:54.918773 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 06 04:17:54.918773 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 04:17:54.918773 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 04:17:54.918773 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 04:17:54.918773 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 04:17:54.918773 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 04:17:54.918773 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 04:17:54.918773 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 04:17:54.918773 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 04:17:54.918773 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 04:17:54.918773 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 04:17:54.918773 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 04:17:54.918773 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 04:17:54.918773 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 04:17:54.921020 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 04:17:54.921020 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 04:17:54.921020 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 06 04:17:54.921020 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 06 04:17:54.921020 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 06 04:17:54.921020 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 06 04:17:54.921020 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 06 04:17:54.921020 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 06 04:17:54.921020 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 06 04:17:54.921020 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 04:17:54.921020 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 359/731] 10.222.0.30 () {56 vars in 1117 bytes} [Tue Sep 6 04:17:54 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 04:17:54.924173 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb5a3ba-bfd9-4384-b06d-00c4e4975570 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:55.002400 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77f2ada-89d5-4697-a467-88cbd4684370 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:55.048546 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d2d2e3-fb1e-4090-9751-9a3f08ffc12c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:55.049169 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d2d2e3-fb1e-4090-9751-9a3f08ffc12c None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:55.049470 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d2d2e3-fb1e-4090-9751-9a3f08ffc12c None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:55.049975 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d2d2e3-fb1e-4090-9751-9a3f08ffc12c None admin] PATH_INFO: `/v3/domains` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:55.050968 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d2d2e3-fb1e-4090-9751-9a3f08ffc12c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:17:55.051273 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d2d2e3-fb1e-4090-9751-9a3f08ffc12c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:17:55.052918 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d2d2e3-fb1e-4090-9751-9a3f08ffc12c None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:17:55.073545 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 373/732] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Sep 6 04:17:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:17:55.075365 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d2d2e3-fb1e-4090-9751-9a3f08ffc12c None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:55.169524 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb5a3ba-bfd9-4384-b06d-00c4e4975570 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:55.218008 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:17:55.230779 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d490b957-205a-452e-b620-2e73f3c04d00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:55.241156 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d490b957-205a-452e-b620-2e73f3c04d00 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:55.241156 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d490b957-205a-452e-b620-2e73f3c04d00 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:55.241156 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d490b957-205a-452e-b620-2e73f3c04d00 None admin] PATH_INFO: `/v3/users` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:55.241156 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d490b957-205a-452e-b620-2e73f3c04d00 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:17:55.241156 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d490b957-205a-452e-b620-2e73f3c04d00 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:17:55.241156 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d490b957-205a-452e-b620-2e73f3c04d00 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:17:55.242126 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d2d2e3-fb1e-4090-9751-9a3f08ffc12c None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:55.259461 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 360/733] 10.222.0.30 () {56 vars in 1149 bytes} [Tue Sep 6 04:17:54 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 336 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 04:17:55.261193 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87189461-0c5f-46b7-9778-0d4e96574588 None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:55.261860 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87189461-0c5f-46b7-9778-0d4e96574588 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:55.262340 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87189461-0c5f-46b7-9778-0d4e96574588 None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:55.267995 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 361/734] 10.222.0.30 () {54 vars in 927 bytes} [Tue Sep 6 04:17:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 04:17:55.269395 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87189461-0c5f-46b7-9778-0d4e96574588 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:55.274268 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:17:55.290948 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7991c4fd-7ee9-44ea-9829-6b9688d284cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:55.291593 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7991c4fd-7ee9-44ea-9829-6b9688d284cf None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:55.291870 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7991c4fd-7ee9-44ea-9829-6b9688d284cf None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:55.292138 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7991c4fd-7ee9-44ea-9829-6b9688d284cf None admin] PATH_INFO: `/v3/users/swift` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:55.302238 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7991c4fd-7ee9-44ea-9829-6b9688d284cf None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:17:55.303998 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7991c4fd-7ee9-44ea-9829-6b9688d284cf None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:17:55.322517 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-7991c4fd-7ee9-44ea-9829-6b9688d284cf None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 06 04:17:55.322517 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 04:17:55.322517 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 04:17:55.322517 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 04:17:55.322517 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 04:17:55.322517 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 04:17:55.322517 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 04:17:55.322517 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 04:17:55.322517 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 04:17:55.322517 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 04:17:55.322517 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 04:17:55.322517 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 04:17:55.322517 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 06 04:17:55.322517 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 06 04:17:55.322517 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 06 04:17:55.322517 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 06 04:17:55.323508 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 04:17:55.323508 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 04:17:55.323508 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 06 04:17:55.323508 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 04:17:55.323508 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 06 04:17:55.323508 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 04:17:55.323508 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 04:17:55.323508 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 04:17:55.323508 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 04:17:55.323508 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 04:17:55.323508 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 04:17:55.323508 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 04:17:55.323508 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 04:17:55.323508 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 04:17:55.323508 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 04:17:55.323508 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 04:17:55.323508 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 04:17:55.323508 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 04:17:55.324466 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 04:17:55.324466 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 04:17:55.324466 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 06 04:17:55.324466 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 06 04:17:55.324466 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 06 04:17:55.324466 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 06 04:17:55.324466 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 06 04:17:55.324466 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 06 04:17:55.324466 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 06 04:17:55.324466 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 04:17:55.324466 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 374/735] 10.222.0.30 () {56 vars in 1096 bytes} [Tue Sep 6 04:17:55 2022] GET /identity/v3/users/swift => generated 83 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 04:17:55.325102 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1b8021-bccd-479f-af84-830114c0cf48 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:55.325724 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1b8021-bccd-479f-af84-830114c0cf48 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:55.325724 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1b8021-bccd-479f-af84-830114c0cf48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:55.351812 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-cb1b8021-bccd-479f-af84-830114c0cf48 None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:17:55.354862 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-cb1b8021-bccd-479f-af84-830114c0cf48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:17:55.400820 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87189461-0c5f-46b7-9778-0d4e96574588 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:55.438826 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc972728-a2ae-4035-9b64-54b2f27bb99b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:55.439489 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc972728-a2ae-4035-9b64-54b2f27bb99b None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:55.439770 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc972728-a2ae-4035-9b64-54b2f27bb99b None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:55.440048 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc972728-a2ae-4035-9b64-54b2f27bb99b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:55.444907 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 375/736] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Sep 6 04:17:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 04:17:55.445994 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1b8021-bccd-479f-af84-830114c0cf48 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:55.519934 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:17:55.532691 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc972728-a2ae-4035-9b64-54b2f27bb99b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:17:55.534835 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc972728-a2ae-4035-9b64-54b2f27bb99b None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:17:55.538746 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:17:55.554882 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1b8021-bccd-479f-af84-830114c0cf48 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:55.591688 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b18ab0-fabf-4bd7-8b94-5b146909d3ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:55.592300 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b18ab0-fabf-4bd7-8b94-5b146909d3ba None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:55.592582 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b18ab0-fabf-4bd7-8b94-5b146909d3ba None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:55.592853 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b18ab0-fabf-4bd7-8b94-5b146909d3ba None admin] PATH_INFO: `/v3/users` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:55.593621 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b18ab0-fabf-4bd7-8b94-5b146909d3ba None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:17:55.593924 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b18ab0-fabf-4bd7-8b94-5b146909d3ba None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:17:55.595371 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b18ab0-fabf-4bd7-8b94-5b146909d3ba None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:17:55.613046 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 376/737] 10.222.0.30 () {56 vars in 1135 bytes} [Tue Sep 6 04:17:55 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:17:55.614357 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d41c9e-3722-4bf4-91a1-aaf4b413d7b4 None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:55.614646 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d41c9e-3722-4bf4-91a1-aaf4b413d7b4 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:55.614920 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d41c9e-3722-4bf4-91a1-aaf4b413d7b4 None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:55.616319 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 377/738] 10.222.0.30 () {54 vars in 927 bytes} [Tue Sep 6 04:17:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 04:17:55.617249 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f30bab-e10a-4588-8c4d-1b96a8f126bb None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:55.617603 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f30bab-e10a-4588-8c4d-1b96a8f126bb None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:55.617926 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f30bab-e10a-4588-8c4d-1b96a8f126bb None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:55.619118 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 378/739] 10.222.0.30 () {54 vars in 927 bytes} [Tue Sep 6 04:17:55 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 04:17:55.619935 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f30bab-e10a-4588-8c4d-1b96a8f126bb None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:55.653531 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 362/740] 10.222.0.30 () {58 vars in 1282 bytes} [Tue Sep 6 04:17:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 06 04:17:55.653531 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e55d8d-536a-4af2-9f7d-db2e0eddf731 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:55.653531 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e55d8d-536a-4af2-9f7d-db2e0eddf731 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:55.653531 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e55d8d-536a-4af2-9f7d-db2e0eddf731 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:55.669821 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-d4e55d8d-536a-4af2-9f7d-db2e0eddf731 None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:17:55.670993 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-d4e55d8d-536a-4af2-9f7d-db2e0eddf731 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:17:55.732525 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f30bab-e10a-4588-8c4d-1b96a8f126bb None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:55.740708 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 363/741] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Sep 6 04:17:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 04:17:55.741982 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f9447a-e7fd-481e-86f2-c026722d2716 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:55.742262 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f9447a-e7fd-481e-86f2-c026722d2716 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:55.742529 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f9447a-e7fd-481e-86f2-c026722d2716 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:55.760730 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-55f9447a-e7fd-481e-86f2-c026722d2716 None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:17:55.761938 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-55f9447a-e7fd-481e-86f2-c026722d2716 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:17:55.771186 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:17:55.779599 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab51796-6f3c-4763-ae95-b44905008b0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:55.780206 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab51796-6f3c-4763-ae95-b44905008b0a None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:55.780485 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab51796-6f3c-4763-ae95-b44905008b0a None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:55.780758 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab51796-6f3c-4763-ae95-b44905008b0a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:55.791484 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:17:55.832281 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 364/742] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Sep 6 04:17:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 04:17:55.833394 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f9447a-e7fd-481e-86f2-c026722d2716 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:55.905793 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab51796-6f3c-4763-ae95-b44905008b0a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:17:55.907774 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab51796-6f3c-4763-ae95-b44905008b0a None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:17:55.945515 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f9447a-e7fd-481e-86f2-c026722d2716 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:55.991890 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe72a9a-7cff-471b-a11c-46bc20928e0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:55.992847 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe72a9a-7cff-471b-a11c-46bc20928e0d None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:55.993338 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe72a9a-7cff-471b-a11c-46bc20928e0d None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:55.993785 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe72a9a-7cff-471b-a11c-46bc20928e0d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:56.007919 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe72a9a-7cff-471b-a11c-46bc20928e0d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:17:56.010205 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe72a9a-7cff-471b-a11c-46bc20928e0d None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:17:56.022524 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 379/743] 10.222.0.30 () {58 vars in 1282 bytes} [Tue Sep 6 04:17:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 403 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 06 04:17:56.023489 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab51796-6f3c-4763-ae95-b44905008b0a None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:56.026884 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-bbe72a9a-7cff-471b-a11c-46bc20928e0d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 04:17:56.026884 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 04:17:56.026884 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 04:17:56.026884 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 04:17:56.026884 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 04:17:56.026884 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 04:17:56.026884 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 04:17:56.026884 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 04:17:56.026884 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 04:17:56.026884 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 04:17:56.026884 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 04:17:56.026884 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 04:17:56.026884 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 04:17:56.026884 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 04:17:56.026884 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 04:17:56.026884 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 04:17:56.027832 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 04:17:56.027832 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 04:17:56.027832 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 04:17:56.027832 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 04:17:56.027832 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 04:17:56.027832 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 04:17:56.027832 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 04:17:56.027832 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 04:17:56.027832 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 04:17:56.027832 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 04:17:56.027832 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 04:17:56.027832 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 04:17:56.027832 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 04:17:56.027832 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 04:17:56.027832 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 04:17:56.027832 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 04:17:56.027832 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 04:17:56.029376 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 04:17:56.029376 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 04:17:56.029376 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 04:17:56.029376 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 365/744] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Sep 6 04:17:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 04:17:56.029376 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe72a9a-7cff-471b-a11c-46bc20928e0d None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:56.048222 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:17:56.055301 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:17:56.131447 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab51796-6f3c-4763-ae95-b44905008b0a None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:56.132062 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe72a9a-7cff-471b-a11c-46bc20928e0d None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:56.165394 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641c146f-83a3-457a-bd38-11de5d28b537 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:56.165610 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e95aaf-d89c-4f8c-94ff-1e1f9a94e1a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:56.166011 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641c146f-83a3-457a-bd38-11de5d28b537 None admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:56.166148 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e95aaf-d89c-4f8c-94ff-1e1f9a94e1a6 None admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:56.166291 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641c146f-83a3-457a-bd38-11de5d28b537 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:56.166429 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e95aaf-d89c-4f8c-94ff-1e1f9a94e1a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:56.166563 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641c146f-83a3-457a-bd38-11de5d28b537 None admin] PATH_INFO: `/v3/services` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:56.166711 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e95aaf-d89c-4f8c-94ff-1e1f9a94e1a6 None admin] PATH_INFO: `/v3/services` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:56.167224 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-641c146f-83a3-457a-bd38-11de5d28b537 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:17:56.167344 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e95aaf-d89c-4f8c-94ff-1e1f9a94e1a6 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:17:56.168599 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-641c146f-83a3-457a-bd38-11de5d28b537 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:17:56.168702 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e95aaf-d89c-4f8c-94ff-1e1f9a94e1a6 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:17:56.246994 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 380/745] 10.222.0.30 () {60 vars in 1141 bytes} [Tue Sep 6 04:17:56 2022] POST /identity/v3/services => generated 240 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 04:17:56.247731 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641c146f-83a3-457a-bd38-11de5d28b537 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:56.260183 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 366/746] 10.222.0.30 () {60 vars in 1141 bytes} [Tue Sep 6 04:17:56 2022] POST /identity/v3/services => generated 251 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 04:17:56.260885 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e95aaf-d89c-4f8c-94ff-1e1f9a94e1a6 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:56.399924 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:17:56.416968 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:17:56.428492 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641c146f-83a3-457a-bd38-11de5d28b537 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:56.467657 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e95aaf-d89c-4f8c-94ff-1e1f9a94e1a6 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:56.476735 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139e55bc-a0eb-412b-948f-c1fcfd268162 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:56.477442 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139e55bc-a0eb-412b-948f-c1fcfd268162 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:56.477728 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139e55bc-a0eb-412b-948f-c1fcfd268162 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:56.477999 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139e55bc-a0eb-412b-948f-c1fcfd268162 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:56.490052 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139e55bc-a0eb-412b-948f-c1fcfd268162 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:17:56.491541 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139e55bc-a0eb-412b-948f-c1fcfd268162 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:17:56.503991 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73602c5f-77fb-49ba-b1b2-4731ba4ab8e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:56.504558 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73602c5f-77fb-49ba-b1b2-4731ba4ab8e5 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:56.504835 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73602c5f-77fb-49ba-b1b2-4731ba4ab8e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:56.505105 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73602c5f-77fb-49ba-b1b2-4731ba4ab8e5 None admin] PATH_INFO: `/v3/domains` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:56.505793 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73602c5f-77fb-49ba-b1b2-4731ba4ab8e5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:17:56.506095 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73602c5f-77fb-49ba-b1b2-4731ba4ab8e5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:17:56.507455 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73602c5f-77fb-49ba-b1b2-4731ba4ab8e5 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:17:56.508288 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-139e55bc-a0eb-412b-948f-c1fcfd268162 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 04:17:56.508288 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 04:17:56.508288 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 04:17:56.508288 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 04:17:56.508288 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 04:17:56.508288 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 04:17:56.508288 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 04:17:56.508288 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 04:17:56.508288 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 04:17:56.508288 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 04:17:56.508288 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 04:17:56.508288 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 04:17:56.508288 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 04:17:56.508288 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 04:17:56.508288 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 04:17:56.508288 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 04:17:56.509274 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 04:17:56.509274 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 04:17:56.509274 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 04:17:56.509274 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 04:17:56.509274 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 04:17:56.509274 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 04:17:56.509274 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 04:17:56.509274 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 04:17:56.509274 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 04:17:56.509274 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 04:17:56.509274 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 04:17:56.509274 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 04:17:56.509274 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 04:17:56.509274 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 04:17:56.509274 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 04:17:56.509274 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 04:17:56.509274 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 04:17:56.510198 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 04:17:56.510198 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 04:17:56.510198 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 04:17:56.510734 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 381/747] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Sep 6 04:17:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 04:17:56.511202 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139e55bc-a0eb-412b-948f-c1fcfd268162 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:56.526774 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 367/748] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Sep 6 04:17:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:17:56.528076 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73602c5f-77fb-49ba-b1b2-4731ba4ab8e5 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:56.651494 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139e55bc-a0eb-412b-948f-c1fcfd268162 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:56.670782 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73602c5f-77fb-49ba-b1b2-4731ba4ab8e5 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:56.703518 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0bb9412-e917-4801-9cc4-764e2a906296 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:56.704691 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bb9412-e917-4801-9cc4-764e2a906296 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:56.704976 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bb9412-e917-4801-9cc4-764e2a906296 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:56.705251 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bb9412-e917-4801-9cc4-764e2a906296 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:56.708321 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:17:56.717782 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:17:56.721162 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0bb9412-e917-4801-9cc4-764e2a906296 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 04:17:56.724555 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af26c6ec-c1d3-42ce-a91d-d78b793cf592 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:56.725447 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af26c6ec-c1d3-42ce-a91d-d78b793cf592 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:56.725909 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af26c6ec-c1d3-42ce-a91d-d78b793cf592 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:56.726360 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af26c6ec-c1d3-42ce-a91d-d78b793cf592 None admin] PATH_INFO: `/v3/domains` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:56.726908 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-b0bb9412-e917-4801-9cc4-764e2a906296 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 04:17:56.726908 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 04:17:56.726908 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 04:17:56.726908 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 04:17:56.726908 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 04:17:56.726908 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 04:17:56.726908 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 04:17:56.726908 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 04:17:56.726908 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 04:17:56.726908 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 04:17:56.726908 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 04:17:56.726908 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 04:17:56.726908 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 04:17:56.726908 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 04:17:56.726908 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 04:17:56.726908 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 04:17:56.727845 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 04:17:56.727845 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 04:17:56.727845 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 04:17:56.727845 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 04:17:56.727845 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 04:17:56.727845 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 04:17:56.727845 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 04:17:56.727845 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 04:17:56.727845 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 04:17:56.727845 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 04:17:56.727845 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 04:17:56.727845 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 04:17:56.727845 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 04:17:56.727845 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 04:17:56.727845 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 04:17:56.727845 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 04:17:56.728685 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 04:17:56.728685 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 04:17:56.728685 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 04:17:56.728685 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 04:17:56.728685 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 04:17:56.728685 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 04:17:56.728685 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 04:17:56.728685 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 04:17:56.728685 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 04:17:56.728685 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 04:17:56.728685 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 04:17:56.728685 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 04:17:56.728685 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 382/749] 10.222.0.30 () {56 vars in 1096 bytes} [Tue Sep 6 04:17:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 04:17:56.729520 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0bb9412-e917-4801-9cc4-764e2a906296 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:56.730268 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af26c6ec-c1d3-42ce-a91d-d78b793cf592 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:17:56.730835 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af26c6ec-c1d3-42ce-a91d-d78b793cf592 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:17:56.732656 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af26c6ec-c1d3-42ce-a91d-d78b793cf592 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:17:56.753438 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 368/750] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Sep 6 04:17:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:17:56.753438 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af26c6ec-c1d3-42ce-a91d-d78b793cf592 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:56.879800 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0bb9412-e917-4801-9cc4-764e2a906296 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:56.891412 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af26c6ec-c1d3-42ce-a91d-d78b793cf592 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:56.925172 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487fc771-5b77-4c43-ac30-be1eb5fb50a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:56.926522 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487fc771-5b77-4c43-ac30-be1eb5fb50a2 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:56.927182 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487fc771-5b77-4c43-ac30-be1eb5fb50a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:56.927603 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487fc771-5b77-4c43-ac30-be1eb5fb50a2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:56.934407 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ef0ba0-0aa6-4eda-89c5-b8c3222d544e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:56.935374 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ef0ba0-0aa6-4eda-89c5-b8c3222d544e None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:56.935734 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ef0ba0-0aa6-4eda-89c5-b8c3222d544e None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:56.936080 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ef0ba0-0aa6-4eda-89c5-b8c3222d544e None admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:56.936401 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487fc771-5b77-4c43-ac30-be1eb5fb50a2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:17:56.936886 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ef0ba0-0aa6-4eda-89c5-b8c3222d544e None admin] RBAC: Adding query filter params (name=admin) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:17:56.937260 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ef0ba0-0aa6-4eda-89c5-b8c3222d544e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:17:56.938370 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487fc771-5b77-4c43-ac30-be1eb5fb50a2 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:17:56.938792 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ef0ba0-0aa6-4eda-89c5-b8c3222d544e None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:17:56.951637 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 369/751] 10.222.0.30 () {56 vars in 1099 bytes} [Tue Sep 6 04:17:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:17:56.952708 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ef0ba0-0aa6-4eda-89c5-b8c3222d544e None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:56.957655 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-487fc771-5b77-4c43-ac30-be1eb5fb50a2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 04:17:56.957655 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 04:17:56.957655 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 04:17:56.957655 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 04:17:56.957655 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 04:17:56.957655 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 04:17:56.957655 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 04:17:56.957655 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 04:17:56.957655 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 04:17:56.957655 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 04:17:56.957655 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 04:17:56.957655 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 04:17:56.957655 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 06 04:17:56.957655 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 06 04:17:56.957655 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 06 04:17:56.958757 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 06 04:17:56.958757 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 04:17:56.958757 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 04:17:56.958757 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 04:17:56.958757 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 04:17:56.958757 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 04:17:56.958757 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 04:17:56.958757 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 04:17:56.958757 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 04:17:56.958757 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 04:17:56.958757 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 04:17:56.958757 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 04:17:56.958757 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 04:17:56.958757 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 04:17:56.958757 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 04:17:56.958757 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 04:17:56.958757 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 04:17:56.959837 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 06 04:17:56.959837 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 06 04:17:56.959837 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 06 04:17:56.959837 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 06 04:17:56.959837 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 06 04:17:56.959837 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 06 04:17:56.959837 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 04:17:56.959837 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 04:17:56.960603 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 383/752] 10.222.0.30 () {56 vars in 1111 bytes} [Tue Sep 6 04:17:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 04:17:56.961454 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487fc771-5b77-4c43-ac30-be1eb5fb50a2 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:57.026076 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:17:57.031511 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:17:57.083238 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ef0ba0-0aa6-4eda-89c5-b8c3222d544e None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:57.086794 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487fc771-5b77-4c43-ac30-be1eb5fb50a2 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:57.120263 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2685c042-7a75-4055-b33f-e95921553393 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:57.121002 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2685c042-7a75-4055-b33f-e95921553393 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:57.121374 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2685c042-7a75-4055-b33f-e95921553393 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:57.121743 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2685c042-7a75-4055-b33f-e95921553393 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:57.128654 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a777c44-2de6-4fc3-999e-1ce8238b38aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:57.129229 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2685c042-7a75-4055-b33f-e95921553393 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:17:57.130257 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a777c44-2de6-4fc3-999e-1ce8238b38aa None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:57.130850 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2685c042-7a75-4055-b33f-e95921553393 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:17:57.131142 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a777c44-2de6-4fc3-999e-1ce8238b38aa None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:57.131874 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a777c44-2de6-4fc3-999e-1ce8238b38aa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:57.142888 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-2685c042-7a75-4055-b33f-e95921553393 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 04:17:57.142888 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 04:17:57.142888 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 04:17:57.142888 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 04:17:57.142888 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 04:17:57.142888 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 04:17:57.142888 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 04:17:57.142888 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 04:17:57.142888 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 04:17:57.142888 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 04:17:57.142888 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 04:17:57.142888 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 04:17:57.142888 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 06 04:17:57.142888 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 06 04:17:57.142888 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 06 04:17:57.144632 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 06 04:17:57.144632 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 04:17:57.144632 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 04:17:57.144632 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 04:17:57.144632 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 04:17:57.144632 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 04:17:57.144632 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 04:17:57.144632 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 04:17:57.144632 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 04:17:57.144632 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 04:17:57.144632 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 04:17:57.144632 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 04:17:57.144632 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 04:17:57.144632 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 04:17:57.144632 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 04:17:57.144632 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 04:17:57.144632 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 04:17:57.145738 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 06 04:17:57.145738 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 06 04:17:57.145738 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 06 04:17:57.145738 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 06 04:17:57.145738 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 06 04:17:57.145738 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 06 04:17:57.145738 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 04:17:57.145738 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 04:17:57.146876 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 370/753] 10.222.0.30 () {56 vars in 1111 bytes} [Tue Sep 6 04:17:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 04:17:57.147802 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2685c042-7a75-4055-b33f-e95921553393 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:57.148585 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a777c44-2de6-4fc3-999e-1ce8238b38aa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:17:57.150255 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a777c44-2de6-4fc3-999e-1ce8238b38aa None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:17:57.166383 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-8a777c44-2de6-4fc3-999e-1ce8238b38aa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 04:17:57.166383 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 04:17:57.166383 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 04:17:57.166383 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 04:17:57.166383 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 04:17:57.166383 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 04:17:57.166383 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 04:17:57.166383 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 04:17:57.166383 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 04:17:57.166383 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 04:17:57.166383 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 04:17:57.166383 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 04:17:57.166383 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 04:17:57.166383 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 04:17:57.166383 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 04:17:57.166383 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 04:17:57.167351 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 04:17:57.167351 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 04:17:57.167351 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 04:17:57.167351 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 04:17:57.167351 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 04:17:57.167351 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 04:17:57.167351 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 04:17:57.167351 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 04:17:57.167351 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 04:17:57.167351 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 04:17:57.167351 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 04:17:57.167351 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 04:17:57.167351 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 04:17:57.167351 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 04:17:57.167351 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 04:17:57.167351 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 04:17:57.167351 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 04:17:57.168244 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 04:17:57.168244 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 04:17:57.168244 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 04:17:57.170543 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 384/754] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Sep 6 04:17:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 04:17:57.171506 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a777c44-2de6-4fc3-999e-1ce8238b38aa None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:57.260817 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2685c042-7a75-4055-b33f-e95921553393 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:57.276012 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:17:57.281260 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a777c44-2de6-4fc3-999e-1ce8238b38aa None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:57.289855 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:17:57.298643 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad77fa8b-4897-4352-b3d3-738bb1b5c789 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:57.299594 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad77fa8b-4897-4352-b3d3-738bb1b5c789 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:57.300863 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad77fa8b-4897-4352-b3d3-738bb1b5c789 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:57.301393 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad77fa8b-4897-4352-b3d3-738bb1b5c789 None admin] PATH_INFO: `/v3/projects` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:57.302390 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad77fa8b-4897-4352-b3d3-738bb1b5c789 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:17:57.302896 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad77fa8b-4897-4352-b3d3-738bb1b5c789 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:17:57.304543 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad77fa8b-4897-4352-b3d3-738bb1b5c789 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:17:57.317547 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 371/755] 10.222.0.30 () {56 vars in 1148 bytes} [Tue Sep 6 04:17:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:17:57.318559 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc3d366-a35d-4f45-811a-e682f4ef8aa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:57.318926 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad77fa8b-4897-4352-b3d3-738bb1b5c789 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:57.319237 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc3d366-a35d-4f45-811a-e682f4ef8aa3 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:57.319519 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc3d366-a35d-4f45-811a-e682f4ef8aa3 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:57.319850 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc3d366-a35d-4f45-811a-e682f4ef8aa3 None admin] PATH_INFO: `/v3/projects` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:57.320605 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc3d366-a35d-4f45-811a-e682f4ef8aa3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:17:57.320914 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc3d366-a35d-4f45-811a-e682f4ef8aa3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:17:57.322509 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc3d366-a35d-4f45-811a-e682f4ef8aa3 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:17:57.337401 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 385/756] 10.222.0.30 () {56 vars in 1148 bytes} [Tue Sep 6 04:17:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:17:57.338373 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc3d366-a35d-4f45-811a-e682f4ef8aa3 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:57.431242 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad77fa8b-4897-4352-b3d3-738bb1b5c789 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:57.450937 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc3d366-a35d-4f45-811a-e682f4ef8aa3 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:57.471404 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700b66ab-689d-4a97-ba69-f7da8a44d6ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:57.472777 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700b66ab-689d-4a97-ba69-f7da8a44d6ae None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:57.473155 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700b66ab-689d-4a97-ba69-f7da8a44d6ae None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:57.473562 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700b66ab-689d-4a97-ba69-f7da8a44d6ae None admin] PATH_INFO: `/v3/domains` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:57.474336 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700b66ab-689d-4a97-ba69-f7da8a44d6ae None admin] RBAC: Adding query filter params (name=Default) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:17:57.474748 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700b66ab-689d-4a97-ba69-f7da8a44d6ae None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:17:57.476274 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700b66ab-689d-4a97-ba69-f7da8a44d6ae None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:17:57.494509 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 372/757] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Sep 6 04:17:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:17:57.495527 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700b66ab-689d-4a97-ba69-f7da8a44d6ae None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:57.497423 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc67f30-2980-4581-adab-b8b933ca72a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:57.498084 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc67f30-2980-4581-adab-b8b933ca72a7 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:57.498367 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc67f30-2980-4581-adab-b8b933ca72a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:57.498636 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc67f30-2980-4581-adab-b8b933ca72a7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:57.499426 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc67f30-2980-4581-adab-b8b933ca72a7 None admin] RBAC: Adding query filter params (role.id=83cf0a194f8d45e792411e2d9afdfe50, scope.project.id=b707733ce65c42edbc1fd95ad0a1c06c, user.id=da5fed17101a4935bcc38b72caf9d263) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:17:57.499771 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc67f30-2980-4581-adab-b8b933ca72a7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:17:57.501284 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc67f30-2980-4581-adab-b8b933ca72a7 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:17:57.508482 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 386/758] 10.222.0.30 () {56 vars in 1374 bytes} [Tue Sep 6 04:17:57 2022] GET /identity/v3/role_assignments?user.id=da5fed17101a4935bcc38b72caf9d263&scope.project.id=b707733ce65c42edbc1fd95ad0a1c06c&role.id=83cf0a194f8d45e792411e2d9afdfe50 => generated 259 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:17:57.509350 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc67f30-2980-4581-adab-b8b933ca72a7 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:57.535552 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:17:57.544622 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:17:57.634864 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700b66ab-689d-4a97-ba69-f7da8a44d6ae None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:57.635426 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc67f30-2980-4581-adab-b8b933ca72a7 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:57.678256 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18238cc-fdc9-4d39-a9d7-f28d8cc0c562 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:57.679011 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18238cc-fdc9-4d39-a9d7-f28d8cc0c562 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:57.679299 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18238cc-fdc9-4d39-a9d7-f28d8cc0c562 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:57.679578 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18238cc-fdc9-4d39-a9d7-f28d8cc0c562 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:57.680328 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18238cc-fdc9-4d39-a9d7-f28d8cc0c562 None admin] RBAC: Adding query filter params (role.id=ea4c814713b149108e488de855b025e4, scope.project.id=b707733ce65c42edbc1fd95ad0a1c06c, user.id=a676ec7f92a144edbeca8e6029ba9ee6) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:17:57.680642 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18238cc-fdc9-4d39-a9d7-f28d8cc0c562 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:17:57.682321 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18238cc-fdc9-4d39-a9d7-f28d8cc0c562 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:17:57.691035 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 373/759] 10.222.0.30 () {56 vars in 1374 bytes} [Tue Sep 6 04:17:57 2022] GET /identity/v3/role_assignments?user.id=a676ec7f92a144edbeca8e6029ba9ee6&scope.project.id=b707733ce65c42edbc1fd95ad0a1c06c&role.id=ea4c814713b149108e488de855b025e4 => generated 259 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:17:57.721096 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657be0d4-b8ae-48d4-8b13-c0273af8eabf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:57.721826 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657be0d4-b8ae-48d4-8b13-c0273af8eabf None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:57.722102 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657be0d4-b8ae-48d4-8b13-c0273af8eabf None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:57.722373 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657be0d4-b8ae-48d4-8b13-c0273af8eabf None admin] PATH_INFO: `/v3/users/nova` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:57.730888 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657be0d4-b8ae-48d4-8b13-c0273af8eabf None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:17:57.732370 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657be0d4-b8ae-48d4-8b13-c0273af8eabf None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:17:57.746238 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-657be0d4-b8ae-48d4-8b13-c0273af8eabf None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 06 04:17:57.746238 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 04:17:57.746238 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 04:17:57.746238 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 04:17:57.746238 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 04:17:57.746238 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 04:17:57.746238 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 04:17:57.746238 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 04:17:57.746238 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 04:17:57.746238 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 04:17:57.746238 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 04:17:57.746238 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 04:17:57.746238 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 06 04:17:57.746238 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 06 04:17:57.746238 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 06 04:17:57.746238 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 06 04:17:57.747283 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 04:17:57.747283 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 04:17:57.747283 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 06 04:17:57.747283 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 04:17:57.747283 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 06 04:17:57.747283 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 04:17:57.747283 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 04:17:57.747283 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 04:17:57.747283 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 04:17:57.747283 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 04:17:57.747283 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 04:17:57.747283 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 04:17:57.747283 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 04:17:57.747283 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 04:17:57.747283 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 04:17:57.747283 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 04:17:57.747283 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 04:17:57.747283 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 04:17:57.748733 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 04:17:57.748733 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 04:17:57.748733 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 06 04:17:57.748733 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 06 04:17:57.748733 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 06 04:17:57.748733 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 06 04:17:57.748733 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 06 04:17:57.748733 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 06 04:17:57.748733 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 06 04:17:57.748733 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 04:17:57.748733 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 387/760] 10.222.0.30 () {56 vars in 1093 bytes} [Tue Sep 6 04:17:57 2022] GET /identity/v3/users/nova => generated 82 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 04:17:57.756507 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18238cc-fdc9-4d39-a9d7-f28d8cc0c562 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:57.906849 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:17:57.911927 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18238cc-fdc9-4d39-a9d7-f28d8cc0c562 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:57.948651 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00df447-b466-40e0-bdca-abf3207ad1d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:57.949195 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00df447-b466-40e0-bdca-abf3207ad1d1 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:57.949854 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00df447-b466-40e0-bdca-abf3207ad1d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:57.950128 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00df447-b466-40e0-bdca-abf3207ad1d1 None admin] PATH_INFO: `/v3/users` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:57.950917 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00df447-b466-40e0-bdca-abf3207ad1d1 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:17:57.951388 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00df447-b466-40e0-bdca-abf3207ad1d1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:17:57.953159 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00df447-b466-40e0-bdca-abf3207ad1d1 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:17:57.972722 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 374/761] 10.222.0.30 () {56 vars in 1133 bytes} [Tue Sep 6 04:17:57 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:17:57.982442 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657be0d4-b8ae-48d4-8b13-c0273af8eabf None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:58.112399 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:17:58.127301 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657be0d4-b8ae-48d4-8b13-c0273af8eabf None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:58.184264 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72089843-78d5-4093-bca7-e2e9d6978316 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:58.185015 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72089843-78d5-4093-bca7-e2e9d6978316 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:58.185321 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72089843-78d5-4093-bca7-e2e9d6978316 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:58.185605 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72089843-78d5-4093-bca7-e2e9d6978316 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:58.201192 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72089843-78d5-4093-bca7-e2e9d6978316 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:17:58.202629 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72089843-78d5-4093-bca7-e2e9d6978316 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:17:58.330981 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-72089843-78d5-4093-bca7-e2e9d6978316 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 04:17:58.330981 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 04:17:58.330981 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 04:17:58.330981 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 04:17:58.330981 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 04:17:58.330981 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 04:17:58.330981 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 04:17:58.330981 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 04:17:58.330981 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 04:17:58.330981 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 04:17:58.330981 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 04:17:58.330981 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 04:17:58.330981 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 04:17:58.330981 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 04:17:58.330981 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 04:17:58.330981 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 04:17:58.332455 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 04:17:58.332455 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 04:17:58.332455 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 04:17:58.332455 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 04:17:58.332455 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 04:17:58.332455 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 04:17:58.332455 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 04:17:58.332455 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 04:17:58.332455 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 04:17:58.332455 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 04:17:58.332455 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 04:17:58.332455 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 04:17:58.332455 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 04:17:58.332455 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 04:17:58.332455 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 04:17:58.332455 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 04:17:58.332455 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 04:17:58.333392 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 04:17:58.333392 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 04:17:58.333392 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 04:17:58.333392 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 388/762] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Sep 6 04:17:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 350 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 04:17:58.337797 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00df447-b466-40e0-bdca-abf3207ad1d1 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:58.488470 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90236f3a-c93f-4b54-b191-68f1aa516823 None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:58.489220 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90236f3a-c93f-4b54-b191-68f1aa516823 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:58.489736 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90236f3a-c93f-4b54-b191-68f1aa516823 None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:58.490548 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00df447-b466-40e0-bdca-abf3207ad1d1 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:58.491870 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 389/763] 10.222.0.30 () {54 vars in 927 bytes} [Tue Sep 6 04:17:58 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 06 04:17:58.497380 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972c7452-441b-4dd8-ad12-928ed9cdf913 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:58.497922 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972c7452-441b-4dd8-ad12-928ed9cdf913 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:58.498479 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972c7452-441b-4dd8-ad12-928ed9cdf913 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:58.518955 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-972c7452-441b-4dd8-ad12-928ed9cdf913 None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:17:58.520516 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-972c7452-441b-4dd8-ad12-928ed9cdf913 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:17:58.580247 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73fd7ab1-ee61-4851-b0d4-d8e806e57a54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:58.580953 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fd7ab1-ee61-4851-b0d4-d8e806e57a54 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:58.581240 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fd7ab1-ee61-4851-b0d4-d8e806e57a54 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:58.581541 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fd7ab1-ee61-4851-b0d4-d8e806e57a54 None admin] PATH_INFO: `/v3/domains` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:58.582217 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73fd7ab1-ee61-4851-b0d4-d8e806e57a54 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:17:58.582791 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73fd7ab1-ee61-4851-b0d4-d8e806e57a54 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:17:58.583875 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73fd7ab1-ee61-4851-b0d4-d8e806e57a54 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:17:58.599431 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 390/764] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Sep 6 04:17:58 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 06 04:17:58.604544 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 375/765] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Sep 6 04:17:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:17:58.607377 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972c7452-441b-4dd8-ad12-928ed9cdf913 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:58.683925 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:17:58.724635 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972c7452-441b-4dd8-ad12-928ed9cdf913 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:58.731583 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfd1ca9-01a3-432e-a390-7c8c7df0a322 None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:58.731873 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfd1ca9-01a3-432e-a390-7c8c7df0a322 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:58.732154 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfd1ca9-01a3-432e-a390-7c8c7df0a322 None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:58.733589 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 376/766] 10.222.0.30 () {54 vars in 927 bytes} [Tue Sep 6 04:17:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 04:17:58.738831 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35670b4f-dd13-4eef-b96c-9b85bbd9d5e5 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:58.739057 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35670b4f-dd13-4eef-b96c-9b85bbd9d5e5 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:58.739333 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35670b4f-dd13-4eef-b96c-9b85bbd9d5e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:58.776150 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-35670b4f-dd13-4eef-b96c-9b85bbd9d5e5 None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:17:58.777386 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-35670b4f-dd13-4eef-b96c-9b85bbd9d5e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:17:58.796864 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:17:58.798985 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4f561e-1b5c-4133-9081-56b7c985c6b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:58.800178 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4f561e-1b5c-4133-9081-56b7c985c6b0 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:58.800681 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4f561e-1b5c-4133-9081-56b7c985c6b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:58.801118 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4f561e-1b5c-4133-9081-56b7c985c6b0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:58.825571 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4f561e-1b5c-4133-9081-56b7c985c6b0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:17:58.827072 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4f561e-1b5c-4133-9081-56b7c985c6b0 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:17:58.846340 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-fd4f561e-1b5c-4133-9081-56b7c985c6b0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 04:17:58.846340 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 04:17:58.846340 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 04:17:58.846340 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 04:17:58.846340 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 04:17:58.846340 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 04:17:58.846340 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 04:17:58.846340 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 04:17:58.846340 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 04:17:58.846340 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 04:17:58.846340 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 04:17:58.846340 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 04:17:58.846340 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 06 04:17:58.846340 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 06 04:17:58.846340 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 06 04:17:58.847948 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 06 04:17:58.847948 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 04:17:58.847948 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 04:17:58.847948 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 04:17:58.847948 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 04:17:58.847948 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 04:17:58.847948 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 04:17:58.847948 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 04:17:58.847948 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 04:17:58.847948 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 04:17:58.847948 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 04:17:58.847948 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 04:17:58.847948 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 04:17:58.847948 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 04:17:58.847948 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 04:17:58.847948 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 04:17:58.847948 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 04:17:58.848849 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 06 04:17:58.848849 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 06 04:17:58.848849 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 06 04:17:58.848849 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 06 04:17:58.848849 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 06 04:17:58.848849 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 06 04:17:58.848849 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 04:17:58.848849 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 04:17:58.848849 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 391/767] 10.222.0.30 () {56 vars in 1111 bytes} [Tue Sep 6 04:17:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 04:17:58.856728 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4f561e-1b5c-4133-9081-56b7c985c6b0 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:58.927904 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 377/768] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Sep 6 04:17:58 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 06 04:17:58.964842 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35670b4f-dd13-4eef-b96c-9b85bbd9d5e5 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:59.033166 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4f561e-1b5c-4133-9081-56b7c985c6b0 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:59.100143 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:17:59.111258 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27783ed-a0ad-423d-a76c-8878d111bd2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:59.111888 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27783ed-a0ad-423d-a76c-8878d111bd2e None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:59.113162 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27783ed-a0ad-423d-a76c-8878d111bd2e None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:59.113162 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27783ed-a0ad-423d-a76c-8878d111bd2e None admin] PATH_INFO: `/v3/projects` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:59.113365 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27783ed-a0ad-423d-a76c-8878d111bd2e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:17:59.113681 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27783ed-a0ad-423d-a76c-8878d111bd2e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:17:59.116091 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27783ed-a0ad-423d-a76c-8878d111bd2e None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:17:59.137330 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 392/769] 10.222.0.30 () {56 vars in 1148 bytes} [Tue Sep 6 04:17:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:17:59.144238 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27783ed-a0ad-423d-a76c-8878d111bd2e None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:59.151375 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35670b4f-dd13-4eef-b96c-9b85bbd9d5e5 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:59.215657 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2400c306-49a4-432b-bfa0-9cf3aaa8f6c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:59.216307 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2400c306-49a4-432b-bfa0-9cf3aaa8f6c5 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:59.216587 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2400c306-49a4-432b-bfa0-9cf3aaa8f6c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:59.216859 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2400c306-49a4-432b-bfa0-9cf3aaa8f6c5 None admin] PATH_INFO: `/v3/services/network` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:59.217501 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2400c306-49a4-432b-bfa0-9cf3aaa8f6c5 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:17:59.218867 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2400c306-49a4-432b-bfa0-9cf3aaa8f6c5 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:17:59.231141 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-2400c306-49a4-432b-bfa0-9cf3aaa8f6c5 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 06 04:17:59.231141 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 04:17:59.231141 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 06 04:17:59.231141 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 06 04:17:59.231141 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 06 04:17:59.231141 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 06 04:17:59.231141 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 06 04:17:59.231141 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 04:17:59.231141 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 06 04:17:59.231141 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 04:17:59.231141 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 06 04:17:59.231141 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 04:17:59.231141 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 04:17:59.231141 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 04:17:59.231141 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 04:17:59.231141 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 04:17:59.231141 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 04:17:59.232163 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 04:17:59.232163 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 04:17:59.232163 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 04:17:59.232163 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 04:17:59.232163 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 04:17:59.232163 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 04:17:59.232163 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 06 04:17:59.232163 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 06 04:17:59.232163 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 06 04:17:59.232163 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 06 04:17:59.232163 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 04:17:59.232163 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 04:17:59.232163 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 04:17:59.232163 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 04:17:59.232163 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 04:17:59.232163 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 04:17:59.233021 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 04:17:59.233021 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 04:17:59.233021 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 04:17:59.233021 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 04:17:59.233021 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 04:17:59.233021 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 04:17:59.233021 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 04:17:59.233021 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 04:17:59.233021 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 04:17:59.233021 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 04:17:59.233021 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 06 04:17:59.233021 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 04:17:59.233021 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 06 04:17:59.233021 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 04:17:59.237405 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 378/770] 10.222.0.30 () {56 vars in 1111 bytes} [Tue Sep 6 04:17:58 2022] GET /identity/v3/services/network => generated 88 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 04:17:59.238499 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2400c306-49a4-432b-bfa0-9cf3aaa8f6c5 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:59.258098 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:17:59.261292 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27783ed-a0ad-423d-a76c-8878d111bd2e None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:59.299388 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c914c17b-5565-4287-91a3-0e10d740c1ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:59.300954 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c914c17b-5565-4287-91a3-0e10d740c1ca None admin] REQUEST_METHOD: `PUT` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:59.301331 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c914c17b-5565-4287-91a3-0e10d740c1ca None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:59.301691 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c914c17b-5565-4287-91a3-0e10d740c1ca None admin] PATH_INFO: `/v3/projects/b707733ce65c42edbc1fd95ad0a1c06c/users/9db138b77afa4b068244b8ef715404fe/roles/ea4c814713b149108e488de855b025e4` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:59.304272 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c914c17b-5565-4287-91a3-0e10d740c1ca None admin] RBAC: Authorizing `identity:create_grant(project_id=b707733ce65c42edbc1fd95ad0a1c06c, user_id=9db138b77afa4b068244b8ef715404fe, role_id=ea4c814713b149108e488de855b025e4)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:17:59.306357 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c914c17b-5565-4287-91a3-0e10d740c1ca None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:17:59.357447 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2400c306-49a4-432b-bfa0-9cf3aaa8f6c5 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:59.378354 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 393/771] 10.222.0.30 () {58 vars in 1439 bytes} [Tue Sep 6 04:17:59 2022] PUT /identity/v3/projects/b707733ce65c42edbc1fd95ad0a1c06c/users/9db138b77afa4b068244b8ef715404fe/roles/ea4c814713b149108e488de855b025e4 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 04:17:59.380550 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c914c17b-5565-4287-91a3-0e10d740c1ca None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:59.400768 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0aa582d-e56b-414a-9afc-024b5fca3c9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:59.401736 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0aa582d-e56b-414a-9afc-024b5fca3c9e None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:59.402273 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0aa582d-e56b-414a-9afc-024b5fca3c9e None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:59.402696 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0aa582d-e56b-414a-9afc-024b5fca3c9e None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:59.403656 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0aa582d-e56b-414a-9afc-024b5fca3c9e None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:17:59.405568 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0aa582d-e56b-414a-9afc-024b5fca3c9e None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:17:59.416304 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-e0aa582d-e56b-414a-9afc-024b5fca3c9e None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 06 04:17:59.416304 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 04:17:59.416304 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 06 04:17:59.416304 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 06 04:17:59.416304 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 06 04:17:59.416304 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 06 04:17:59.416304 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 06 04:17:59.416304 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 04:17:59.416304 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 06 04:17:59.416304 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 04:17:59.416304 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 06 04:17:59.416304 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 04:17:59.416304 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 04:17:59.416304 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 04:17:59.416304 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 04:17:59.416304 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 04:17:59.416304 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 04:17:59.417459 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 04:17:59.417459 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 04:17:59.417459 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 04:17:59.417459 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 04:17:59.417459 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 04:17:59.417459 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 04:17:59.417459 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 06 04:17:59.417459 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 06 04:17:59.417459 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 06 04:17:59.417459 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 06 04:17:59.417459 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 04:17:59.417459 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 04:17:59.417459 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 04:17:59.417459 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 04:17:59.417459 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 04:17:59.417459 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 04:17:59.418330 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 04:17:59.418330 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 04:17:59.418330 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 04:17:59.418330 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 04:17:59.418330 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 04:17:59.418330 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 04:17:59.418330 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 04:17:59.418330 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 04:17:59.418330 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 04:17:59.418330 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 04:17:59.418330 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 06 04:17:59.418330 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 04:17:59.418330 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 06 04:17:59.418330 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 04:17:59.421147 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 379/772] 10.222.0.30 () {56 vars in 1129 bytes} [Tue Sep 6 04:17:59 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 04:17:59.428095 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0aa582d-e56b-414a-9afc-024b5fca3c9e None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:59.452098 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:17:59.507312 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c914c17b-5565-4287-91a3-0e10d740c1ca None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:59.534311 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:17:59.548695 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464e16b9-83d4-43b4-b906-0a0cd1d4f458 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:59.549701 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464e16b9-83d4-43b4-b906-0a0cd1d4f458 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:59.550131 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464e16b9-83d4-43b4-b906-0a0cd1d4f458 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:59.550702 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464e16b9-83d4-43b4-b906-0a0cd1d4f458 None admin] PATH_INFO: `/v3/services` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:59.551487 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464e16b9-83d4-43b4-b906-0a0cd1d4f458 None admin] RBAC: Adding query filter params (name=network) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:17:59.551958 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464e16b9-83d4-43b4-b906-0a0cd1d4f458 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:17:59.553518 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464e16b9-83d4-43b4-b906-0a0cd1d4f458 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:17:59.559046 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 394/773] 10.222.0.30 () {56 vars in 1112 bytes} [Tue Sep 6 04:17:59 2022] GET /identity/v3/services?name=network => generated 124 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:17:59.564960 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464e16b9-83d4-43b4-b906-0a0cd1d4f458 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:59.566896 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0aa582d-e56b-414a-9afc-024b5fca3c9e None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:59.646622 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a16fa9-6eb6-41e2-8aac-c39b0304f2f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:59.648205 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a16fa9-6eb6-41e2-8aac-c39b0304f2f2 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:59.648634 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a16fa9-6eb6-41e2-8aac-c39b0304f2f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:59.649072 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a16fa9-6eb6-41e2-8aac-c39b0304f2f2 None admin] PATH_INFO: `/v3/services` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:59.650415 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a16fa9-6eb6-41e2-8aac-c39b0304f2f2 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:17:59.650889 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a16fa9-6eb6-41e2-8aac-c39b0304f2f2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:17:59.653217 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a16fa9-6eb6-41e2-8aac-c39b0304f2f2 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:17:59.662247 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 380/774] 10.222.0.30 () {56 vars in 1124 bytes} [Tue Sep 6 04:17:59 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:17:59.668057 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a16fa9-6eb6-41e2-8aac-c39b0304f2f2 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:59.731043 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464e16b9-83d4-43b4-b906-0a0cd1d4f458 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:59.756160 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:17:59.789788 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d88ee46-c528-406a-afbc-3e0293b2a6fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:59.790024 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d88ee46-c528-406a-afbc-3e0293b2a6fc None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:59.790305 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d88ee46-c528-406a-afbc-3e0293b2a6fc None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:59.790581 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d88ee46-c528-406a-afbc-3e0293b2a6fc None admin] PATH_INFO: `/v3/services` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:59.791225 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d88ee46-c528-406a-afbc-3e0293b2a6fc None admin] RBAC: Adding query filter params (type=network) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:17:59.791525 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d88ee46-c528-406a-afbc-3e0293b2a6fc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:17:59.792897 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d88ee46-c528-406a-afbc-3e0293b2a6fc None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:17:59.804740 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a16fa9-6eb6-41e2-8aac-c39b0304f2f2 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:59.804902 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 395/775] 10.222.0.30 () {56 vars in 1112 bytes} [Tue Sep 6 04:17:59 2022] GET /identity/v3/services?type=network => generated 350 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 04:17:59.810542 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c186eac-47e4-4acc-b2c2-713b0cd30ece None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:59.810828 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c186eac-47e4-4acc-b2c2-713b0cd30ece None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:59.811101 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c186eac-47e4-4acc-b2c2-713b0cd30ece None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:59.812629 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 396/776] 10.222.0.30 () {54 vars in 927 bytes} [Tue Sep 6 04:17:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 04:17:59.813347 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c186eac-47e4-4acc-b2c2-713b0cd30ece None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:59.851272 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa6c6d0-ab03-47f0-8ec0-35205b8fd567 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:59.851891 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa6c6d0-ab03-47f0-8ec0-35205b8fd567 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:59.852172 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa6c6d0-ab03-47f0-8ec0-35205b8fd567 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:59.852441 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa6c6d0-ab03-47f0-8ec0-35205b8fd567 None admin] PATH_INFO: `/v3/services` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:59.853071 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa6c6d0-ab03-47f0-8ec0-35205b8fd567 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:17:59.854155 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa6c6d0-ab03-47f0-8ec0-35205b8fd567 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:17:59.856134 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa6c6d0-ab03-47f0-8ec0-35205b8fd567 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:17:59.867439 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 381/777] 10.222.0.30 () {56 vars in 1124 bytes} [Tue Sep 6 04:17:59 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 04:17:59.872281 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afe7b31-cd77-4ebd-a136-1167e039e390 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:59.872281 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afe7b31-cd77-4ebd-a136-1167e039e390 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:59.872281 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afe7b31-cd77-4ebd-a136-1167e039e390 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:59.878306 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:17:59.895199 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-9afe7b31-cd77-4ebd-a136-1167e039e390 None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:17:59.897416 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-9afe7b31-cd77-4ebd-a136-1167e039e390 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:17:59.954574 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c186eac-47e4-4acc-b2c2-713b0cd30ece None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:17:59.992409 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 382/778] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Sep 6 04:17:59 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 04:17:59.993948 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9afe7b31-cd77-4ebd-a136-1167e039e390 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:17:59.994954 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d842f3ca-abac-494d-b1da-6cf7fe628bd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:17:59.995685 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d842f3ca-abac-494d-b1da-6cf7fe628bd1 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:17:59.996040 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d842f3ca-abac-494d-b1da-6cf7fe628bd1 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:17:59.996384 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d842f3ca-abac-494d-b1da-6cf7fe628bd1 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:17:59.997334 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d842f3ca-abac-494d-b1da-6cf7fe628bd1 None admin] RBAC: Adding query filter params (interface=public, service_id=1999e365490f460bba24441a5abec7d4, region_id=RegionOne) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:17:59.997614 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d842f3ca-abac-494d-b1da-6cf7fe628bd1 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:17:59.999264 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d842f3ca-abac-494d-b1da-6cf7fe628bd1 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:18:00.008186 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 397/779] 10.222.0.30 () {56 vars in 1251 bytes} [Tue Sep 6 04:17:59 2022] GET /identity/v3/endpoints?service_id=1999e365490f460bba24441a5abec7d4&interface=public®ion_id=RegionOne => generated 194 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:18:00.013580 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d842f3ca-abac-494d-b1da-6cf7fe628bd1 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:18:00.066717 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:18:00.120130 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9afe7b31-cd77-4ebd-a136-1167e039e390 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:18:00.138349 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d842f3ca-abac-494d-b1da-6cf7fe628bd1 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:18:00.153960 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:18:00.160316 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5265c96-3b06-4481-bd0b-5949a8d6259f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:18:00.161076 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5265c96-3b06-4481-bd0b-5949a8d6259f None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:18:00.161501 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5265c96-3b06-4481-bd0b-5949a8d6259f None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:18:00.161842 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5265c96-3b06-4481-bd0b-5949a8d6259f None admin] PATH_INFO: `/v3/endpoints` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:18:00.162561 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5265c96-3b06-4481-bd0b-5949a8d6259f None admin] RBAC: Adding query filter params (interface=public, service_id=f06bc3f0860048ce9ffdc06aa175cf5e, region_id=RegionOne) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:18:00.162871 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5265c96-3b06-4481-bd0b-5949a8d6259f None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:18:00.164443 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5265c96-3b06-4481-bd0b-5949a8d6259f None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:18:00.177331 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 383/780] 10.222.0.30 () {56 vars in 1251 bytes} [Tue Sep 6 04:17:59 2022] GET /identity/v3/endpoints?service_id=f06bc3f0860048ce9ffdc06aa175cf5e&interface=public®ion_id=RegionOne => generated 194 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:18:00.178842 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433cf690-0ee1-4ee4-aaaa-4850c1a9e126 None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:18:00.179179 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433cf690-0ee1-4ee4-aaaa-4850c1a9e126 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:18:00.179560 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433cf690-0ee1-4ee4-aaaa-4850c1a9e126 None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:18:00.180887 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a68ece4-ca77-4136-b3b4-5764b46b71fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:18:00.181176 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 384/781] 10.222.0.30 () {54 vars in 927 bytes} [Tue Sep 6 04:18:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 04:18:00.181930 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a68ece4-ca77-4136-b3b4-5764b46b71fc None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:18:00.182377 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a68ece4-ca77-4136-b3b4-5764b46b71fc None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:18:00.182828 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a68ece4-ca77-4136-b3b4-5764b46b71fc None admin] PATH_INFO: `/v3/services` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:18:00.183617 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a68ece4-ca77-4136-b3b4-5764b46b71fc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:18:00.184366 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433cf690-0ee1-4ee4-aaaa-4850c1a9e126 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:18:00.185504 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a68ece4-ca77-4136-b3b4-5764b46b71fc None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:18:00.193155 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 398/782] 10.222.0.30 () {56 vars in 1087 bytes} [Tue Sep 6 04:18:00 2022] GET /identity/v3/services => generated 772 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 04:18:00.194599 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf984957-a085-418d-8f53-5493f2ed276b None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:18:00.195057 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf984957-a085-418d-8f53-5493f2ed276b None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:18:00.195528 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf984957-a085-418d-8f53-5493f2ed276b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:18:00.225599 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-cf984957-a085-418d-8f53-5493f2ed276b None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:18:00.231072 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-cf984957-a085-418d-8f53-5493f2ed276b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:18:00.344651 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 399/783] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Sep 6 04:18:00 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 04:18:00.345643 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf984957-a085-418d-8f53-5493f2ed276b None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:18:00.351018 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433cf690-0ee1-4ee4-aaaa-4850c1a9e126 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:18:00.422404 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:18:00.423490 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b3771f-82c6-4161-9f7e-66e28dff9272 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:18:00.424749 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b3771f-82c6-4161-9f7e-66e28dff9272 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:18:00.425333 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b3771f-82c6-4161-9f7e-66e28dff9272 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:18:00.425815 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b3771f-82c6-4161-9f7e-66e28dff9272 None admin] PATH_INFO: `/v3/services` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:18:00.426696 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b3771f-82c6-4161-9f7e-66e28dff9272 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:18:00.428267 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b3771f-82c6-4161-9f7e-66e28dff9272 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:18:00.434199 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 385/784] 10.222.0.30 () {56 vars in 1087 bytes} [Tue Sep 6 04:18:00 2022] GET /identity/v3/services => generated 772 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:18:00.503263 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf984957-a085-418d-8f53-5493f2ed276b None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:18:00.544891 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c0cd5d-6280-4fd7-bacc-1417ead79f5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:18:00.545632 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c0cd5d-6280-4fd7-bacc-1417ead79f5b None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:18:00.545916 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c0cd5d-6280-4fd7-bacc-1417ead79f5b None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:18:00.546186 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c0cd5d-6280-4fd7-bacc-1417ead79f5b None admin] PATH_INFO: `/v3/roles/service` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:18:00.559783 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c0cd5d-6280-4fd7-bacc-1417ead79f5b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 04:18:00.564829 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-c9c0cd5d-6280-4fd7-bacc-1417ead79f5b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 06 04:18:00.564829 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 04:18:00.564829 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 04:18:00.564829 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 04:18:00.564829 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 04:18:00.564829 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 04:18:00.564829 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 04:18:00.564829 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 04:18:00.564829 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 04:18:00.564829 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 04:18:00.564829 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 04:18:00.564829 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 04:18:00.564829 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 04:18:00.564829 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 04:18:00.564829 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 04:18:00.564829 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 04:18:00.565882 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 04:18:00.565882 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 04:18:00.565882 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 04:18:00.565882 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 04:18:00.565882 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 04:18:00.565882 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 04:18:00.565882 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 04:18:00.565882 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 04:18:00.565882 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 04:18:00.565882 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 04:18:00.565882 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 04:18:00.565882 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 04:18:00.565882 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 04:18:00.565882 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 04:18:00.565882 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 04:18:00.565882 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 04:18:00.566745 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 04:18:00.566745 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 04:18:00.566745 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 04:18:00.566745 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 04:18:00.566745 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 04:18:00.566745 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 04:18:00.566745 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 04:18:00.566745 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 04:18:00.566745 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 04:18:00.566745 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 04:18:00.566745 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 06 04:18:00.566745 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 04:18:00.567385 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 400/785] 10.222.0.30 () {56 vars in 1102 bytes} [Tue Sep 6 04:18:00 2022] GET /identity/v3/roles/service => generated 85 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 04:18:00.572348 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b3771f-82c6-4161-9f7e-66e28dff9272 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:18:00.639842 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:18:00.640035 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c0cd5d-6280-4fd7-bacc-1417ead79f5b None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:18:00.702346 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b3771f-82c6-4161-9f7e-66e28dff9272 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:18:00.761812 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ee3bb6-c9d9-4bad-835b-1cb8e9966adf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:18:00.762430 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ee3bb6-c9d9-4bad-835b-1cb8e9966adf None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:18:00.762712 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ee3bb6-c9d9-4bad-835b-1cb8e9966adf None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:18:00.762983 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ee3bb6-c9d9-4bad-835b-1cb8e9966adf None admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:18:00.763662 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ee3bb6-c9d9-4bad-835b-1cb8e9966adf None admin] RBAC: Adding query filter params (name=service) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:18:00.763962 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ee3bb6-c9d9-4bad-835b-1cb8e9966adf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:18:00.765346 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ee3bb6-c9d9-4bad-835b-1cb8e9966adf None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:18:00.782934 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 386/786] 10.222.0.30 () {56 vars in 1103 bytes} [Tue Sep 6 04:18:00 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 04:18:00.788923 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ee3bb6-c9d9-4bad-835b-1cb8e9966adf None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:18:00.792199 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:18:00.803093 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c0cd5d-6280-4fd7-bacc-1417ead79f5b None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:18:00.875567 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5348014-96e7-4a02-b842-613cac0e1d58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:18:00.876220 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5348014-96e7-4a02-b842-613cac0e1d58 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:18:00.876502 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5348014-96e7-4a02-b842-613cac0e1d58 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:18:00.876773 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5348014-96e7-4a02-b842-613cac0e1d58 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:18:00.902543 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5348014-96e7-4a02-b842-613cac0e1d58 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 04:18:00.907564 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-c5348014-96e7-4a02-b842-613cac0e1d58 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 04:18:00.907564 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 04:18:00.907564 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 04:18:00.907564 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 04:18:00.907564 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 04:18:00.907564 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 04:18:00.907564 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 04:18:00.907564 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 04:18:00.907564 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 04:18:00.907564 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 04:18:00.907564 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 04:18:00.907564 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 04:18:00.907564 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 04:18:00.907564 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 04:18:00.907564 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 04:18:00.907564 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 04:18:00.908517 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 04:18:00.908517 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 04:18:00.908517 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 04:18:00.908517 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 04:18:00.908517 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 04:18:00.908517 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 04:18:00.908517 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 04:18:00.908517 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 04:18:00.908517 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 04:18:00.908517 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 04:18:00.908517 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 04:18:00.908517 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 04:18:00.908517 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 04:18:00.908517 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 04:18:00.908517 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 04:18:00.908517 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 04:18:00.909614 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 04:18:00.909614 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 04:18:00.909614 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 04:18:00.909614 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 04:18:00.909614 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 04:18:00.909614 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 04:18:00.909614 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 04:18:00.909614 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 04:18:00.909614 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 04:18:00.909614 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 04:18:00.909614 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 04:18:00.909614 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 04:18:00.909614 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 401/787] 10.222.0.30 () {56 vars in 1096 bytes} [Tue Sep 6 04:18:00 2022] GET /identity/v3/roles/admin => generated 83 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 04:18:00.927173 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5348014-96e7-4a02-b842-613cac0e1d58 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:18:00.932657 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ee3bb6-c9d9-4bad-835b-1cb8e9966adf None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:18:00.956774 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:18:00.969257 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d161342-906a-4f19-ae47-784bce257310 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:18:00.970111 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d161342-906a-4f19-ae47-784bce257310 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:18:00.982915 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d161342-906a-4f19-ae47-784bce257310 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:18:00.982915 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d161342-906a-4f19-ae47-784bce257310 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:18:00.983471 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d161342-906a-4f19-ae47-784bce257310 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:18:00.985026 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d161342-906a-4f19-ae47-784bce257310 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:18:01.001170 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-9d161342-906a-4f19-ae47-784bce257310 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 04:18:01.001170 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 04:18:01.001170 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 04:18:01.001170 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 04:18:01.001170 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 04:18:01.001170 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 04:18:01.001170 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 04:18:01.001170 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 04:18:01.001170 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 04:18:01.001170 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 04:18:01.001170 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 04:18:01.001170 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 04:18:01.001170 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 04:18:01.001170 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 04:18:01.001170 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 04:18:01.001170 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 04:18:01.004078 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 04:18:01.004078 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 04:18:01.004078 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 04:18:01.004078 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 04:18:01.004078 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 04:18:01.004078 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 04:18:01.004078 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 04:18:01.004078 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 04:18:01.004078 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 04:18:01.004078 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 04:18:01.004078 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 04:18:01.004078 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 04:18:01.004078 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 04:18:01.004078 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 04:18:01.004078 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 04:18:01.004078 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 04:18:01.004078 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 04:18:01.005751 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 04:18:01.005751 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 04:18:01.005751 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 04:18:01.005751 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 387/788] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Sep 6 04:18:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 04:18:01.015723 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d161342-906a-4f19-ae47-784bce257310 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:18:01.130468 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5348014-96e7-4a02-b842-613cac0e1d58 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:18:01.204697 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178d6bee-11cf-4136-8449-23f911831d79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:18:01.205240 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178d6bee-11cf-4136-8449-23f911831d79 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:18:01.205546 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178d6bee-11cf-4136-8449-23f911831d79 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:18:01.205911 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178d6bee-11cf-4136-8449-23f911831d79 None admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:18:01.206586 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-178d6bee-11cf-4136-8449-23f911831d79 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:18:01.206953 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-178d6bee-11cf-4136-8449-23f911831d79 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:18:01.208259 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-178d6bee-11cf-4136-8449-23f911831d79 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:18:01.222951 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 402/789] 10.222.0.30 () {56 vars in 1099 bytes} [Tue Sep 6 04:18:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:18:01.230769 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178d6bee-11cf-4136-8449-23f911831d79 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:18:01.266033 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:18:01.309876 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d161342-906a-4f19-ae47-784bce257310 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:18:01.345912 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e3c4fd-c655-4984-a643-994967e776a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:18:01.347199 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e3c4fd-c655-4984-a643-994967e776a6 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:18:01.347549 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e3c4fd-c655-4984-a643-994967e776a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:18:01.347890 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e3c4fd-c655-4984-a643-994967e776a6 None admin] PATH_INFO: `/v3/domains` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:18:01.348612 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e3c4fd-c655-4984-a643-994967e776a6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:18:01.348981 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e3c4fd-c655-4984-a643-994967e776a6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:18:01.350459 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e3c4fd-c655-4984-a643-994967e776a6 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:18:01.363202 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 388/790] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Sep 6 04:18:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 347 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:18:01.374402 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e3c4fd-c655-4984-a643-994967e776a6 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:18:01.407029 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178d6bee-11cf-4136-8449-23f911831d79 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:18:01.450001 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31695965-7525-4414-82b6-b49f86f21b70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:18:01.450617 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31695965-7525-4414-82b6-b49f86f21b70 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:18:01.450900 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31695965-7525-4414-82b6-b49f86f21b70 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:18:01.451235 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31695965-7525-4414-82b6-b49f86f21b70 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:18:01.462255 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:18:01.476242 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31695965-7525-4414-82b6-b49f86f21b70 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:18:01.477785 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31695965-7525-4414-82b6-b49f86f21b70 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:18:01.520259 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-31695965-7525-4414-82b6-b49f86f21b70 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 04:18:01.520259 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 04:18:01.520259 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 04:18:01.520259 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 04:18:01.520259 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 04:18:01.520259 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 04:18:01.520259 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 04:18:01.520259 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 04:18:01.520259 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 04:18:01.520259 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 04:18:01.520259 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 04:18:01.520259 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 04:18:01.520259 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 04:18:01.520259 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 04:18:01.520259 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 04:18:01.520259 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 04:18:01.521284 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 04:18:01.521284 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 04:18:01.521284 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 04:18:01.521284 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 04:18:01.521284 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 04:18:01.521284 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 04:18:01.521284 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 04:18:01.521284 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 04:18:01.521284 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 04:18:01.521284 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 04:18:01.521284 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 04:18:01.521284 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 04:18:01.521284 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 04:18:01.521284 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 04:18:01.521284 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 04:18:01.521284 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 04:18:01.521284 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 04:18:01.522223 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 04:18:01.522223 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 04:18:01.522223 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 04:18:01.525235 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 403/791] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Sep 6 04:18:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 295 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 04:18:01.527787 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31695965-7525-4414-82b6-b49f86f21b70 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:18:01.584877 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e3c4fd-c655-4984-a643-994967e776a6 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:18:01.661603 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84aaab3d-8d0e-47ed-a3bc-02e6c498568a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:18:01.662244 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84aaab3d-8d0e-47ed-a3bc-02e6c498568a None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:18:01.662523 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84aaab3d-8d0e-47ed-a3bc-02e6c498568a None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:18:01.662791 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84aaab3d-8d0e-47ed-a3bc-02e6c498568a None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:18:01.676189 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84aaab3d-8d0e-47ed-a3bc-02e6c498568a None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:18:01.677669 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84aaab3d-8d0e-47ed-a3bc-02e6c498568a None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:18:01.692924 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-84aaab3d-8d0e-47ed-a3bc-02e6c498568a None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 06 04:18:01.692924 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 04:18:01.692924 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 04:18:01.692924 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 04:18:01.692924 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 04:18:01.692924 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 04:18:01.692924 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 04:18:01.692924 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 04:18:01.692924 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 04:18:01.692924 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 04:18:01.692924 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 04:18:01.692924 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 04:18:01.692924 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 06 04:18:01.692924 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 06 04:18:01.692924 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 06 04:18:01.692924 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 06 04:18:01.693965 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 04:18:01.693965 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 04:18:01.693965 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 06 04:18:01.693965 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 04:18:01.693965 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 06 04:18:01.693965 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 04:18:01.693965 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 04:18:01.693965 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 04:18:01.693965 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 04:18:01.693965 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 04:18:01.693965 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 04:18:01.693965 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 04:18:01.693965 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 04:18:01.693965 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 04:18:01.693965 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 04:18:01.693965 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 04:18:01.693965 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 04:18:01.693965 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 04:18:01.694919 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 04:18:01.694919 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 04:18:01.694919 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 06 04:18:01.694919 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 06 04:18:01.694919 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 06 04:18:01.694919 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 06 04:18:01.694919 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 06 04:18:01.694919 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 06 04:18:01.694919 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 06 04:18:01.694919 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 04:18:01.694919 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 389/792] 10.222.0.30 () {56 vars in 1117 bytes} [Tue Sep 6 04:18:01 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 321 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 04:18:01.700484 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:18:01.702191 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84aaab3d-8d0e-47ed-a3bc-02e6c498568a None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:18:01.766810 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31695965-7525-4414-82b6-b49f86f21b70 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:18:01.841582 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2edee88-adeb-4350-88b8-eacb96220b7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:18:01.842477 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2edee88-adeb-4350-88b8-eacb96220b7f None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:18:01.842909 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2edee88-adeb-4350-88b8-eacb96220b7f None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:18:01.843363 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2edee88-adeb-4350-88b8-eacb96220b7f None admin] PATH_INFO: `/v3/domains` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:18:01.844185 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2edee88-adeb-4350-88b8-eacb96220b7f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:18:01.844643 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2edee88-adeb-4350-88b8-eacb96220b7f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:18:01.846247 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2edee88-adeb-4350-88b8-eacb96220b7f None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:18:01.867845 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 404/793] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Sep 6 04:18:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 340 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:18:01.870058 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6db126-14a6-46f4-808e-19042df7ff01 None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:18:01.870522 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6db126-14a6-46f4-808e-19042df7ff01 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:18:01.871096 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:18:01.872636 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6db126-14a6-46f4-808e-19042df7ff01 None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:18:01.874335 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 405/794] 10.222.0.30 () {54 vars in 927 bytes} [Tue Sep 6 04:18:01 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 04:18:01.875357 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6db126-14a6-46f4-808e-19042df7ff01 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:18:01.883469 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84aaab3d-8d0e-47ed-a3bc-02e6c498568a None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:18:01.921278 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58fbe29f-6768-4c46-a508-e68f1bb12ae9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:18:01.922176 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fbe29f-6768-4c46-a508-e68f1bb12ae9 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:18:01.922579 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fbe29f-6768-4c46-a508-e68f1bb12ae9 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:18:01.922953 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fbe29f-6768-4c46-a508-e68f1bb12ae9 None admin] PATH_INFO: `/v3/users` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:18:01.923912 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58fbe29f-6768-4c46-a508-e68f1bb12ae9 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:18:01.924318 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58fbe29f-6768-4c46-a508-e68f1bb12ae9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:18:01.926111 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58fbe29f-6768-4c46-a508-e68f1bb12ae9 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:18:01.946889 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 390/795] 10.222.0.30 () {56 vars in 1149 bytes} [Tue Sep 6 04:18:01 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:18:01.948208 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f875c8-0fb5-4d35-9979-ad7ba7bb79b5 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:18:01.948525 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f875c8-0fb5-4d35-9979-ad7ba7bb79b5 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:18:01.948803 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f875c8-0fb5-4d35-9979-ad7ba7bb79b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:18:01.968527 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-f7f875c8-0fb5-4d35-9979-ad7ba7bb79b5 None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:18:01.969766 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-f7f875c8-0fb5-4d35-9979-ad7ba7bb79b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:18:02.002138 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6db126-14a6-46f4-808e-19042df7ff01 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:18:02.036669 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:18:02.042355 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdcf0253-2477-4e3f-9a8e-9332648cb567 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:18:02.043333 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcf0253-2477-4e3f-9a8e-9332648cb567 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:18:02.043814 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcf0253-2477-4e3f-9a8e-9332648cb567 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:18:02.044251 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcf0253-2477-4e3f-9a8e-9332648cb567 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:18:02.051230 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 391/796] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Sep 6 04:18:01 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 04:18:02.051230 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f875c8-0fb5-4d35-9979-ad7ba7bb79b5 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:18:02.059198 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdcf0253-2477-4e3f-9a8e-9332648cb567 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:18:02.060873 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdcf0253-2477-4e3f-9a8e-9332648cb567 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:18:02.073974 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-fdcf0253-2477-4e3f-9a8e-9332648cb567 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 06 04:18:02.073974 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 04:18:02.073974 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 04:18:02.073974 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 04:18:02.073974 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 04:18:02.073974 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 04:18:02.073974 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 04:18:02.073974 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 04:18:02.073974 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 04:18:02.073974 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 04:18:02.073974 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 04:18:02.073974 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 04:18:02.073974 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 06 04:18:02.073974 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 06 04:18:02.073974 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 06 04:18:02.073974 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 06 04:18:02.074971 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 04:18:02.074971 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 04:18:02.074971 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 06 04:18:02.074971 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 04:18:02.074971 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 06 04:18:02.074971 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 04:18:02.074971 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 04:18:02.074971 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 04:18:02.074971 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 04:18:02.074971 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 04:18:02.074971 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 04:18:02.074971 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 04:18:02.074971 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 04:18:02.074971 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 04:18:02.074971 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 04:18:02.074971 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 04:18:02.074971 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 04:18:02.074971 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 04:18:02.075914 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 04:18:02.075914 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 04:18:02.075914 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 06 04:18:02.075914 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 06 04:18:02.075914 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 06 04:18:02.075914 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 06 04:18:02.075914 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 06 04:18:02.075914 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 06 04:18:02.075914 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 06 04:18:02.075914 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 04:18:02.078155 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 406/797] 10.222.0.30 () {56 vars in 1096 bytes} [Tue Sep 6 04:18:01 2022] GET /identity/v3/users/swift => generated 83 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 04:18:02.090496 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdcf0253-2477-4e3f-9a8e-9332648cb567 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:18:02.199110 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdcf0253-2477-4e3f-9a8e-9332648cb567 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:18:02.235223 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b650f74-9351-4bba-96ea-1c61e7aadda3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:18:02.235941 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b650f74-9351-4bba-96ea-1c61e7aadda3 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:18:02.236281 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b650f74-9351-4bba-96ea-1c61e7aadda3 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:18:02.236556 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b650f74-9351-4bba-96ea-1c61e7aadda3 None admin] PATH_INFO: `/v3/users` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:18:02.237887 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b650f74-9351-4bba-96ea-1c61e7aadda3 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:18:02.238193 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b650f74-9351-4bba-96ea-1c61e7aadda3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:18:02.239617 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b650f74-9351-4bba-96ea-1c61e7aadda3 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:18:02.242353 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:18:02.260502 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 407/798] 10.222.0.30 () {56 vars in 1135 bytes} [Tue Sep 6 04:18:02 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:18:02.269446 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b650f74-9351-4bba-96ea-1c61e7aadda3 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:18:02.285042 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f875c8-0fb5-4d35-9979-ad7ba7bb79b5 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:18:02.324225 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:18:02.374759 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa9c05e-2b75-41a0-ab01-cd1ae892b6fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:18:02.375644 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa9c05e-2b75-41a0-ab01-cd1ae892b6fb None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:18:02.375920 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa9c05e-2b75-41a0-ab01-cd1ae892b6fb None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:18:02.376475 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa9c05e-2b75-41a0-ab01-cd1ae892b6fb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:18:02.383899 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b650f74-9351-4bba-96ea-1c61e7aadda3 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:18:02.391795 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa9c05e-2b75-41a0-ab01-cd1ae892b6fb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:18:02.394346 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa9c05e-2b75-41a0-ab01-cd1ae892b6fb None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:18:02.413602 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-5aa9c05e-2b75-41a0-ab01-cd1ae892b6fb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 04:18:02.413602 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 04:18:02.413602 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 04:18:02.413602 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 04:18:02.413602 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 04:18:02.413602 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 04:18:02.413602 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 04:18:02.413602 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 04:18:02.413602 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 04:18:02.413602 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 04:18:02.413602 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 04:18:02.413602 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 04:18:02.413602 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 04:18:02.413602 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 04:18:02.413602 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 04:18:02.413602 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 04:18:02.414679 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 04:18:02.414679 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 04:18:02.414679 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 04:18:02.414679 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 04:18:02.414679 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 04:18:02.414679 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 04:18:02.414679 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 04:18:02.414679 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 04:18:02.414679 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 04:18:02.414679 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 04:18:02.414679 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 04:18:02.414679 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 04:18:02.414679 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 04:18:02.414679 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 04:18:02.414679 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 04:18:02.414679 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 04:18:02.414679 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 04:18:02.415652 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 04:18:02.415652 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 04:18:02.415652 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 04:18:02.415835 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 392/799] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Sep 6 04:18:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 370 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 04:18:02.419493 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa9c05e-2b75-41a0-ab01-cd1ae892b6fb None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:18:02.429639 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3280cc6a-0052-462d-b7e0-677ee854b1dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:18:02.430520 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3280cc6a-0052-462d-b7e0-677ee854b1dd None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:18:02.430899 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3280cc6a-0052-462d-b7e0-677ee854b1dd None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:18:02.431533 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3280cc6a-0052-462d-b7e0-677ee854b1dd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:18:02.448779 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3280cc6a-0052-462d-b7e0-677ee854b1dd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:18:02.450605 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3280cc6a-0052-462d-b7e0-677ee854b1dd None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:18:02.469664 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-3280cc6a-0052-462d-b7e0-677ee854b1dd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 04:18:02.469664 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 04:18:02.469664 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 04:18:02.469664 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 04:18:02.469664 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 04:18:02.469664 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 04:18:02.469664 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 04:18:02.469664 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 04:18:02.469664 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 04:18:02.469664 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 04:18:02.469664 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 04:18:02.469664 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 04:18:02.469664 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 04:18:02.469664 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 04:18:02.469664 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 04:18:02.469664 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 04:18:02.470651 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 04:18:02.470651 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 04:18:02.470651 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 04:18:02.470651 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 04:18:02.470651 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 04:18:02.470651 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 04:18:02.470651 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 04:18:02.470651 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 04:18:02.470651 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 04:18:02.470651 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 04:18:02.470651 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 04:18:02.470651 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 04:18:02.470651 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 04:18:02.470651 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 04:18:02.470651 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 04:18:02.470651 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 04:18:02.470651 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 04:18:02.472813 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 04:18:02.472813 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 04:18:02.472813 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 04:18:02.476593 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 408/800] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Sep 6 04:18:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 04:18:02.477502 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3280cc6a-0052-462d-b7e0-677ee854b1dd None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:18:02.550300 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa9c05e-2b75-41a0-ab01-cd1ae892b6fb None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:18:02.591083 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484fae09-cd2a-4091-a880-313669aafce4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:18:02.591636 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484fae09-cd2a-4091-a880-313669aafce4 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:18:02.591911 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484fae09-cd2a-4091-a880-313669aafce4 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:18:02.592202 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484fae09-cd2a-4091-a880-313669aafce4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:18:02.594039 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:18:02.603924 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484fae09-cd2a-4091-a880-313669aafce4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 04:18:02.607916 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3280cc6a-0052-462d-b7e0-677ee854b1dd None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:18:02.608884 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-484fae09-cd2a-4091-a880-313669aafce4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 04:18:02.608884 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 04:18:02.608884 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 04:18:02.608884 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 04:18:02.608884 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 04:18:02.608884 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 04:18:02.608884 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 04:18:02.608884 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 04:18:02.608884 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 04:18:02.608884 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 04:18:02.608884 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 04:18:02.608884 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 04:18:02.608884 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 04:18:02.608884 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 04:18:02.608884 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 04:18:02.608884 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 04:18:02.609850 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 04:18:02.609850 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 04:18:02.609850 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 04:18:02.609850 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 04:18:02.609850 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 04:18:02.609850 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 04:18:02.609850 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 04:18:02.609850 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 04:18:02.609850 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 04:18:02.609850 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 04:18:02.609850 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 04:18:02.609850 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 04:18:02.609850 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 04:18:02.609850 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 04:18:02.609850 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 04:18:02.609850 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 04:18:02.614819 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 04:18:02.614819 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 04:18:02.614819 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 04:18:02.614819 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 04:18:02.614819 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 04:18:02.614819 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 04:18:02.614819 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 04:18:02.614819 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 04:18:02.614819 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 04:18:02.614819 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 04:18:02.614819 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 04:18:02.614819 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 04:18:02.615588 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:18:02.615673 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 393/801] 10.222.0.30 () {56 vars in 1096 bytes} [Tue Sep 6 04:18:02 2022] GET /identity/v3/roles/admin => generated 83 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 04:18:02.615673 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484fae09-cd2a-4091-a880-313669aafce4 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:18:02.651069 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486e964b-4c65-4513-9d22-220e02b07d6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:18:02.651672 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486e964b-4c65-4513-9d22-220e02b07d6c None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:18:02.651948 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486e964b-4c65-4513-9d22-220e02b07d6c None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:18:02.652222 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486e964b-4c65-4513-9d22-220e02b07d6c None admin] PATH_INFO: `/v3/domains` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:18:02.652867 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-486e964b-4c65-4513-9d22-220e02b07d6c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:18:02.653170 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-486e964b-4c65-4513-9d22-220e02b07d6c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:18:02.654612 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-486e964b-4c65-4513-9d22-220e02b07d6c None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:18:02.671807 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 409/802] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Sep 6 04:18:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:18:02.671807 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a89261-73e5-4907-b617-8f28be993e39 None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:18:02.671807 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a89261-73e5-4907-b617-8f28be993e39 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:18:02.671807 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a89261-73e5-4907-b617-8f28be993e39 None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:18:02.671807 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 410/803] 10.222.0.30 () {54 vars in 927 bytes} [Tue Sep 6 04:18:02 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 04:18:02.671807 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a89261-73e5-4907-b617-8f28be993e39 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:18:02.727825 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484fae09-cd2a-4091-a880-313669aafce4 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:18:02.765030 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1f3908-dc08-4353-8819-caae01237361 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:18:02.765734 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1f3908-dc08-4353-8819-caae01237361 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:18:02.766021 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1f3908-dc08-4353-8819-caae01237361 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:18:02.766284 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1f3908-dc08-4353-8819-caae01237361 None admin] PATH_INFO: `/v3/domains` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:18:02.766928 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1f3908-dc08-4353-8819-caae01237361 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:18:02.767396 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1f3908-dc08-4353-8819-caae01237361 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:18:02.769442 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1f3908-dc08-4353-8819-caae01237361 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:18:02.781297 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 394/804] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Sep 6 04:18:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:18:02.782174 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1f3908-dc08-4353-8819-caae01237361 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:18:02.782277 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a89261-73e5-4907-b617-8f28be993e39 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:18:02.818516 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c964bd5-10e6-43b8-8273-4e23492fff16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:18:02.819129 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c964bd5-10e6-43b8-8273-4e23492fff16 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:18:02.819411 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c964bd5-10e6-43b8-8273-4e23492fff16 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:18:02.819686 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c964bd5-10e6-43b8-8273-4e23492fff16 None admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:18:02.820381 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c964bd5-10e6-43b8-8273-4e23492fff16 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:18:02.820684 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c964bd5-10e6-43b8-8273-4e23492fff16 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:18:02.822093 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c964bd5-10e6-43b8-8273-4e23492fff16 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:18:02.830655 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 411/805] 10.222.0.30 () {56 vars in 1099 bytes} [Tue Sep 6 04:18:02 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:18:02.831939 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6345eb76-d0f0-430e-9697-34ae32bae5fc None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:18:02.832222 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6345eb76-d0f0-430e-9697-34ae32bae5fc None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:18:02.832492 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6345eb76-d0f0-430e-9697-34ae32bae5fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:18:02.840347 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:18:02.849717 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-6345eb76-d0f0-430e-9697-34ae32bae5fc None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:18:02.850850 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-6345eb76-d0f0-430e-9697-34ae32bae5fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:18:02.868311 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:18:02.885947 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1f3908-dc08-4353-8819-caae01237361 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:18:02.917049 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 412/806] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Sep 6 04:18:02 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 04:18:02.917995 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21c1f84-2653-4348-a326-03ca2e1b41d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:18:02.918423 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2aa218-3134-4826-99b2-369522618074 None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:18:02.918623 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21c1f84-2653-4348-a326-03ca2e1b41d5 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:18:02.918719 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2aa218-3134-4826-99b2-369522618074 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:18:02.918903 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21c1f84-2653-4348-a326-03ca2e1b41d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:18:02.919003 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2aa218-3134-4826-99b2-369522618074 None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:18:02.919177 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21c1f84-2653-4348-a326-03ca2e1b41d5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:18:02.924353 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 413/807] 10.222.0.30 () {54 vars in 927 bytes} [Tue Sep 6 04:18:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 04:18:02.924353 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2aa218-3134-4826-99b2-369522618074 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:18:02.935687 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21c1f84-2653-4348-a326-03ca2e1b41d5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:18:02.937119 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21c1f84-2653-4348-a326-03ca2e1b41d5 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:18:02.948843 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-f21c1f84-2653-4348-a326-03ca2e1b41d5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 04:18:02.948843 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 04:18:02.948843 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 04:18:02.948843 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 04:18:02.948843 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 04:18:02.948843 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 04:18:02.948843 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 04:18:02.948843 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 04:18:02.948843 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 04:18:02.948843 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 04:18:02.948843 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 04:18:02.948843 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 04:18:02.948843 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 06 04:18:02.948843 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 06 04:18:02.948843 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 06 04:18:02.949783 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 06 04:18:02.949783 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 04:18:02.949783 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 04:18:02.949783 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 04:18:02.949783 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 04:18:02.949783 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 04:18:02.949783 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 04:18:02.949783 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 04:18:02.949783 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 04:18:02.949783 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 04:18:02.949783 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 04:18:02.949783 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 04:18:02.949783 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 04:18:02.949783 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 04:18:02.949783 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 04:18:02.949783 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 04:18:02.949783 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 04:18:02.950675 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 06 04:18:02.950675 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 06 04:18:02.950675 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 06 04:18:02.950675 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 06 04:18:02.950675 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 06 04:18:02.950675 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 06 04:18:02.950675 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 04:18:02.950675 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 04:18:02.951453 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 395/808] 10.222.0.30 () {56 vars in 1111 bytes} [Tue Sep 6 04:18:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 04:18:02.952172 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21c1f84-2653-4348-a326-03ca2e1b41d5 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:18:03.039019 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2aa218-3134-4826-99b2-369522618074 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:18:03.058004 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21c1f84-2653-4348-a326-03ca2e1b41d5 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:18:03.073547 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:18:03.081301 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2cbae5-a968-4722-b25d-fad9bf7d916e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:18:03.081954 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2cbae5-a968-4722-b25d-fad9bf7d916e None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:18:03.082228 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2cbae5-a968-4722-b25d-fad9bf7d916e None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:18:03.087237 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2cbae5-a968-4722-b25d-fad9bf7d916e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:18:03.091399 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2cbae5-a968-4722-b25d-fad9bf7d916e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:18:03.092832 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2cbae5-a968-4722-b25d-fad9bf7d916e None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:18:03.095634 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc275a8c-4e5b-4b74-8b14-796dadc670f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:18:03.096259 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc275a8c-4e5b-4b74-8b14-796dadc670f0 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:18:03.096576 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc275a8c-4e5b-4b74-8b14-796dadc670f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:18:03.096887 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc275a8c-4e5b-4b74-8b14-796dadc670f0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:18:03.104990 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-0e2cbae5-a968-4722-b25d-fad9bf7d916e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 04:18:03.104990 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 04:18:03.104990 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 04:18:03.104990 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 04:18:03.104990 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 04:18:03.104990 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 04:18:03.104990 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 04:18:03.104990 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 04:18:03.104990 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 04:18:03.104990 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 04:18:03.104990 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 04:18:03.104990 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 04:18:03.104990 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 06 04:18:03.104990 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 06 04:18:03.104990 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 06 04:18:03.106063 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 06 04:18:03.106063 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 04:18:03.106063 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 04:18:03.106063 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 04:18:03.106063 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 04:18:03.106063 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 04:18:03.106063 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 04:18:03.106063 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 04:18:03.106063 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 04:18:03.106063 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 04:18:03.106063 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 04:18:03.106063 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 04:18:03.106063 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 04:18:03.106063 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 04:18:03.106063 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 04:18:03.106063 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 04:18:03.106063 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 04:18:03.107055 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 06 04:18:03.107055 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 06 04:18:03.107055 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 06 04:18:03.107055 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 06 04:18:03.107055 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 06 04:18:03.107055 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 06 04:18:03.107055 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 04:18:03.107055 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 04:18:03.109443 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 414/809] 10.222.0.30 () {56 vars in 1111 bytes} [Tue Sep 6 04:18:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 04:18:03.110576 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc275a8c-4e5b-4b74-8b14-796dadc670f0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:18:03.111092 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3327af53-e822-4e00-92a3-98d7f4cef382 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:18:03.111383 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3327af53-e822-4e00-92a3-98d7f4cef382 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:18:03.111660 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3327af53-e822-4e00-92a3-98d7f4cef382 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:18:03.113020 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc275a8c-4e5b-4b74-8b14-796dadc670f0 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:18:03.128210 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-fc275a8c-4e5b-4b74-8b14-796dadc670f0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 04:18:03.128210 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 04:18:03.128210 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 04:18:03.128210 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 04:18:03.128210 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 04:18:03.128210 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 04:18:03.128210 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 04:18:03.128210 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 04:18:03.128210 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 04:18:03.128210 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 04:18:03.128210 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 04:18:03.128210 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 04:18:03.128210 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 04:18:03.128210 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 04:18:03.128210 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 04:18:03.128210 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 04:18:03.129161 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 04:18:03.129161 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 04:18:03.129161 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 04:18:03.129161 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 04:18:03.129161 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 04:18:03.129161 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 04:18:03.129161 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 04:18:03.129161 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 04:18:03.129161 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 04:18:03.129161 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 04:18:03.129161 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 04:18:03.129161 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 04:18:03.129161 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 04:18:03.129161 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 04:18:03.129161 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 04:18:03.129161 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 04:18:03.129161 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 04:18:03.135059 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 04:18:03.135059 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 04:18:03.135059 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 04:18:03.135059 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 396/810] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Sep 6 04:18:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 04:18:03.135059 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc275a8c-4e5b-4b74-8b14-796dadc670f0 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:18:03.135358 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-3327af53-e822-4e00-92a3-98d7f4cef382 None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:18:03.136257 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-3327af53-e822-4e00-92a3-98d7f4cef382 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:18:03.146604 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:18:03.206472 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 415/811] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Sep 6 04:18:03 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 04:18:03.207194 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3327af53-e822-4e00-92a3-98d7f4cef382 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:18:03.236619 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc275a8c-4e5b-4b74-8b14-796dadc670f0 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:18:03.273860 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4123669-24c9-4d09-a838-b17df4b6620a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:18:03.274616 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4123669-24c9-4d09-a838-b17df4b6620a None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:18:03.274898 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4123669-24c9-4d09-a838-b17df4b6620a None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:18:03.275165 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4123669-24c9-4d09-a838-b17df4b6620a None admin] PATH_INFO: `/v3/projects` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:18:03.275850 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4123669-24c9-4d09-a838-b17df4b6620a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:18:03.276144 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4123669-24c9-4d09-a838-b17df4b6620a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:18:03.277584 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4123669-24c9-4d09-a838-b17df4b6620a None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:18:03.293942 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 397/812] 10.222.0.30 () {56 vars in 1148 bytes} [Tue Sep 6 04:18:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:18:03.293942 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4123669-24c9-4d09-a838-b17df4b6620a None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:18:03.317793 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:18:03.335198 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3327af53-e822-4e00-92a3-98d7f4cef382 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:18:03.378055 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817e8e71-5c61-4194-b471-069a8bc0d97a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:18:03.378646 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817e8e71-5c61-4194-b471-069a8bc0d97a None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:18:03.379062 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817e8e71-5c61-4194-b471-069a8bc0d97a None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:18:03.379338 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817e8e71-5c61-4194-b471-069a8bc0d97a None admin] PATH_INFO: `/v3/projects` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:18:03.380034 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-817e8e71-5c61-4194-b471-069a8bc0d97a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:18:03.380334 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-817e8e71-5c61-4194-b471-069a8bc0d97a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:18:03.381792 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-817e8e71-5c61-4194-b471-069a8bc0d97a None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:18:03.395079 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 416/813] 10.222.0.30 () {56 vars in 1148 bytes} [Tue Sep 6 04:18:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:18:03.395696 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817e8e71-5c61-4194-b471-069a8bc0d97a None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:18:03.400042 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:18:03.405188 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4123669-24c9-4d09-a838-b17df4b6620a None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:18:03.440645 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df112dd-1c1c-4bc0-b648-df0edca6a628 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:18:03.441256 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df112dd-1c1c-4bc0-b648-df0edca6a628 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:18:03.441561 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df112dd-1c1c-4bc0-b648-df0edca6a628 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:18:03.441828 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df112dd-1c1c-4bc0-b648-df0edca6a628 None admin] PATH_INFO: `/v3/domains` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:18:03.442466 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df112dd-1c1c-4bc0-b648-df0edca6a628 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:18:03.442765 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df112dd-1c1c-4bc0-b648-df0edca6a628 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:18:03.444121 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df112dd-1c1c-4bc0-b648-df0edca6a628 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:18:03.455583 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 398/814] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Sep 6 04:18:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:18:03.456984 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df112dd-1c1c-4bc0-b648-df0edca6a628 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:18:03.510439 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817e8e71-5c61-4194-b471-069a8bc0d97a None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:18:03.544120 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:18:03.544746 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658e0cbb-b245-4fd1-b6ef-2a3f8736dfbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:18:03.545409 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658e0cbb-b245-4fd1-b6ef-2a3f8736dfbb None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:18:03.545690 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658e0cbb-b245-4fd1-b6ef-2a3f8736dfbb None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:18:03.545962 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658e0cbb-b245-4fd1-b6ef-2a3f8736dfbb None admin] PATH_INFO: `/v3/services/network` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:18:03.546547 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658e0cbb-b245-4fd1-b6ef-2a3f8736dfbb None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:18:03.547915 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658e0cbb-b245-4fd1-b6ef-2a3f8736dfbb None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:18:03.557067 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-658e0cbb-b245-4fd1-b6ef-2a3f8736dfbb None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 06 04:18:03.557067 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 04:18:03.557067 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 06 04:18:03.557067 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 06 04:18:03.557067 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 06 04:18:03.557067 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 06 04:18:03.557067 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 06 04:18:03.557067 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 04:18:03.557067 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 06 04:18:03.557067 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 04:18:03.557067 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 06 04:18:03.557067 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 04:18:03.557067 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 04:18:03.557067 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 04:18:03.557067 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 04:18:03.557067 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 04:18:03.557067 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 04:18:03.562004 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 04:18:03.562004 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 04:18:03.562004 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 04:18:03.562004 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 04:18:03.562004 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 04:18:03.562004 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 04:18:03.562004 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 06 04:18:03.562004 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 06 04:18:03.562004 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 06 04:18:03.562004 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 06 04:18:03.562004 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 04:18:03.562004 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 04:18:03.562004 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 04:18:03.562004 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 04:18:03.562004 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 04:18:03.562004 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 04:18:03.562858 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 04:18:03.562858 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 04:18:03.562858 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 04:18:03.562858 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 04:18:03.562858 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 04:18:03.562858 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 04:18:03.562858 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 04:18:03.562858 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 04:18:03.562858 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 04:18:03.562858 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 04:18:03.562858 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 06 04:18:03.562858 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 04:18:03.562858 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 06 04:18:03.562858 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 04:18:03.562858 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 417/815] 10.222.0.30 () {56 vars in 1111 bytes} [Tue Sep 6 04:18:03 2022] GET /identity/v3/services/network => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 04:18:03.563678 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658e0cbb-b245-4fd1-b6ef-2a3f8736dfbb None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:18:03.563747 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df112dd-1c1c-4bc0-b648-df0edca6a628 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:18:03.597287 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76603016-8537-44f7-bbbb-3769a6ec3017 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:18:03.597975 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76603016-8537-44f7-bbbb-3769a6ec3017 None admin] REQUEST_METHOD: `PUT` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:18:03.598265 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76603016-8537-44f7-bbbb-3769a6ec3017 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:18:03.598545 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76603016-8537-44f7-bbbb-3769a6ec3017 None admin] PATH_INFO: `/v3/projects/b707733ce65c42edbc1fd95ad0a1c06c/users/da5fed17101a4935bcc38b72caf9d263/roles/83cf0a194f8d45e792411e2d9afdfe50` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:18:03.614335 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76603016-8537-44f7-bbbb-3769a6ec3017 None admin] RBAC: Authorizing `identity:create_grant(project_id=b707733ce65c42edbc1fd95ad0a1c06c, user_id=da5fed17101a4935bcc38b72caf9d263, role_id=83cf0a194f8d45e792411e2d9afdfe50)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:18:03.615763 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76603016-8537-44f7-bbbb-3769a6ec3017 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:18:03.625172 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:18:03.665397 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658e0cbb-b245-4fd1-b6ef-2a3f8736dfbb None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:18:03.699013 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845f7f40-4958-4ba8-91a6-14d3ce3d5d88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:18:03.699664 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845f7f40-4958-4ba8-91a6-14d3ce3d5d88 None admin] REQUEST_METHOD: `PUT` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:18:03.699942 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845f7f40-4958-4ba8-91a6-14d3ce3d5d88 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:18:03.700213 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845f7f40-4958-4ba8-91a6-14d3ce3d5d88 None admin] PATH_INFO: `/v3/projects/b707733ce65c42edbc1fd95ad0a1c06c/users/a676ec7f92a144edbeca8e6029ba9ee6/roles/ea4c814713b149108e488de855b025e4` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:18:03.702663 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-845f7f40-4958-4ba8-91a6-14d3ce3d5d88 None admin] RBAC: Authorizing `identity:create_grant(project_id=b707733ce65c42edbc1fd95ad0a1c06c, user_id=a676ec7f92a144edbeca8e6029ba9ee6, role_id=ea4c814713b149108e488de855b025e4)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:18:03.703672 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-845f7f40-4958-4ba8-91a6-14d3ce3d5d88 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:18:03.727241 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 399/816] 10.222.0.30 () {58 vars in 1439 bytes} [Tue Sep 6 04:18:03 2022] PUT /identity/v3/projects/b707733ce65c42edbc1fd95ad0a1c06c/users/da5fed17101a4935bcc38b72caf9d263/roles/83cf0a194f8d45e792411e2d9afdfe50 => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 04:18:03.728086 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76603016-8537-44f7-bbbb-3769a6ec3017 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:18:03.817694 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 418/817] 10.222.0.30 () {58 vars in 1439 bytes} [Tue Sep 6 04:18:03 2022] PUT /identity/v3/projects/b707733ce65c42edbc1fd95ad0a1c06c/users/a676ec7f92a144edbeca8e6029ba9ee6/roles/ea4c814713b149108e488de855b025e4 => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 04:18:03.819088 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845f7f40-4958-4ba8-91a6-14d3ce3d5d88 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:18:03.847999 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76603016-8537-44f7-bbbb-3769a6ec3017 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:18:03.905261 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d3f82b-037f-42d1-87ed-4660d7c7f36b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:18:03.906323 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d3f82b-037f-42d1-87ed-4660d7c7f36b None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:18:03.906785 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d3f82b-037f-42d1-87ed-4660d7c7f36b None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:18:03.907268 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d3f82b-037f-42d1-87ed-4660d7c7f36b None admin] PATH_INFO: `/v3/users/nova` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:18:03.922318 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:18:03.930044 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d3f82b-037f-42d1-87ed-4660d7c7f36b None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:18:03.931938 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d3f82b-037f-42d1-87ed-4660d7c7f36b None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:18:03.948496 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-d5d3f82b-037f-42d1-87ed-4660d7c7f36b None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 06 04:18:03.948496 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 04:18:03.948496 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 04:18:03.948496 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 04:18:03.948496 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 04:18:03.948496 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 04:18:03.948496 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 04:18:03.948496 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 04:18:03.948496 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 04:18:03.948496 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 04:18:03.948496 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 04:18:03.948496 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 04:18:03.948496 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 06 04:18:03.948496 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 06 04:18:03.948496 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 06 04:18:03.948496 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 06 04:18:03.949634 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 04:18:03.949634 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 04:18:03.949634 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 06 04:18:03.949634 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 04:18:03.949634 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 06 04:18:03.949634 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 04:18:03.949634 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 04:18:03.949634 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 04:18:03.949634 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 04:18:03.949634 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 04:18:03.949634 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 04:18:03.949634 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 04:18:03.949634 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 04:18:03.949634 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 04:18:03.949634 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 04:18:03.949634 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 04:18:03.949634 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 04:18:03.949634 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 04:18:03.950583 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 04:18:03.950583 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 04:18:03.950583 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 06 04:18:03.950583 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 06 04:18:03.950583 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 06 04:18:03.950583 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 06 04:18:03.950583 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 06 04:18:03.950583 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 06 04:18:03.950583 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 06 04:18:03.950583 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 04:18:03.951126 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 400/818] 10.222.0.30 () {56 vars in 1093 bytes} [Tue Sep 6 04:18:03 2022] GET /identity/v3/users/nova => generated 82 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 04:18:03.951653 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d3f82b-037f-42d1-87ed-4660d7c7f36b None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:18:03.951761 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845f7f40-4958-4ba8-91a6-14d3ce3d5d88 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:18:03.996852 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:18:04.006440 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24f62d3-3e5f-4120-8444-234d2642a605 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:18:04.007106 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24f62d3-3e5f-4120-8444-234d2642a605 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:18:04.007388 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24f62d3-3e5f-4120-8444-234d2642a605 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:18:04.007695 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24f62d3-3e5f-4120-8444-234d2642a605 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:18:04.008263 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24f62d3-3e5f-4120-8444-234d2642a605 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:18:04.009752 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24f62d3-3e5f-4120-8444-234d2642a605 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:18:04.019652 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-a24f62d3-3e5f-4120-8444-234d2642a605 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 06 04:18:04.019652 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 04:18:04.019652 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 06 04:18:04.019652 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 06 04:18:04.019652 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 06 04:18:04.019652 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 06 04:18:04.019652 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 06 04:18:04.019652 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 04:18:04.019652 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 06 04:18:04.019652 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 04:18:04.019652 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 06 04:18:04.019652 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 04:18:04.019652 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 04:18:04.019652 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 04:18:04.019652 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 04:18:04.019652 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 04:18:04.019652 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 04:18:04.022030 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 04:18:04.022030 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 04:18:04.022030 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 04:18:04.022030 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 04:18:04.022030 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 04:18:04.022030 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 04:18:04.022030 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 06 04:18:04.022030 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 06 04:18:04.022030 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 06 04:18:04.022030 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 06 04:18:04.022030 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 04:18:04.022030 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 04:18:04.022030 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 04:18:04.022030 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 04:18:04.022030 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 04:18:04.022030 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 04:18:04.024096 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 04:18:04.024096 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 04:18:04.024096 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 04:18:04.024096 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 04:18:04.024096 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 04:18:04.024096 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 04:18:04.024096 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 04:18:04.024096 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 04:18:04.024096 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 04:18:04.024096 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 04:18:04.024096 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 06 04:18:04.024096 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 04:18:04.024096 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 06 04:18:04.024096 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 04:18:04.024096 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 419/819] 10.222.0.30 () {56 vars in 1129 bytes} [Tue Sep 6 04:18:03 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 04:18:04.025465 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24f62d3-3e5f-4120-8444-234d2642a605 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:18:04.106326 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d3f82b-037f-42d1-87ed-4660d7c7f36b None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:18:04.136101 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24f62d3-3e5f-4120-8444-234d2642a605 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:18:04.154722 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e301e028-62a9-41bb-ad3a-3216e7a57c00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:18:04.155367 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e301e028-62a9-41bb-ad3a-3216e7a57c00 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:18:04.155644 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e301e028-62a9-41bb-ad3a-3216e7a57c00 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:18:04.155915 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e301e028-62a9-41bb-ad3a-3216e7a57c00 None admin] PATH_INFO: `/v3/services` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:18:04.156550 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e301e028-62a9-41bb-ad3a-3216e7a57c00 None admin] RBAC: Adding query filter params (name=network) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:18:04.156851 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e301e028-62a9-41bb-ad3a-3216e7a57c00 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:18:04.160002 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e301e028-62a9-41bb-ad3a-3216e7a57c00 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:18:04.165339 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 401/820] 10.222.0.30 () {56 vars in 1112 bytes} [Tue Sep 6 04:18:03 2022] GET /identity/v3/services?name=network => generated 124 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:18:04.165995 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e301e028-62a9-41bb-ad3a-3216e7a57c00 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:18:04.179530 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072aa298-5bd0-4baf-8f21-3f724ce5b99a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:18:04.180073 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072aa298-5bd0-4baf-8f21-3f724ce5b99a None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:18:04.180354 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072aa298-5bd0-4baf-8f21-3f724ce5b99a None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:18:04.180629 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072aa298-5bd0-4baf-8f21-3f724ce5b99a None admin] PATH_INFO: `/v3/users` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:18:04.181485 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072aa298-5bd0-4baf-8f21-3f724ce5b99a None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:18:04.181794 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072aa298-5bd0-4baf-8f21-3f724ce5b99a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:18:04.183349 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072aa298-5bd0-4baf-8f21-3f724ce5b99a None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:18:04.207141 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 420/821] 10.222.0.30 () {56 vars in 1133 bytes} [Tue Sep 6 04:18:04 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:18:04.208074 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072aa298-5bd0-4baf-8f21-3f724ce5b99a None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:18:04.241684 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:18:04.263418 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:18:04.283712 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e301e028-62a9-41bb-ad3a-3216e7a57c00 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:18:04.317121 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072aa298-5bd0-4baf-8f21-3f724ce5b99a None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:18:04.322393 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd83830b-859a-44f7-bf77-6436a4576003 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:18:04.323017 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd83830b-859a-44f7-bf77-6436a4576003 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:18:04.323301 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd83830b-859a-44f7-bf77-6436a4576003 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:18:04.323579 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd83830b-859a-44f7-bf77-6436a4576003 None admin] PATH_INFO: `/v3/services` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:18:04.324217 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd83830b-859a-44f7-bf77-6436a4576003 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:18:04.324522 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd83830b-859a-44f7-bf77-6436a4576003 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:18:04.326017 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd83830b-859a-44f7-bf77-6436a4576003 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:18:04.335943 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 402/822] 10.222.0.30 () {56 vars in 1124 bytes} [Tue Sep 6 04:18:04 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:18:04.336921 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd83830b-859a-44f7-bf77-6436a4576003 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:18:04.357844 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc803c88-1dea-47de-bdb0-ac9c8a1e92a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:18:04.358617 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc803c88-1dea-47de-bdb0-ac9c8a1e92a6 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:18:04.359001 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc803c88-1dea-47de-bdb0-ac9c8a1e92a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:18:04.359377 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc803c88-1dea-47de-bdb0-ac9c8a1e92a6 None admin] PATH_INFO: `/v3/services` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:18:04.360175 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc803c88-1dea-47de-bdb0-ac9c8a1e92a6 None admin] RBAC: Adding query filter params (type=network) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:18:04.364536 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc803c88-1dea-47de-bdb0-ac9c8a1e92a6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:18:04.364536 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc803c88-1dea-47de-bdb0-ac9c8a1e92a6 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:18:04.370972 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 421/823] 10.222.0.30 () {56 vars in 1112 bytes} [Tue Sep 6 04:18:04 2022] GET /identity/v3/services?type=network => generated 350 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:18:04.371983 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc803c88-1dea-47de-bdb0-ac9c8a1e92a6 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:18:04.458486 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd83830b-859a-44f7-bf77-6436a4576003 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:18:04.488227 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:18:04.503262 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b7adbe-3214-4a6b-a548-e2e9f0681994 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:18:04.506230 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b7adbe-3214-4a6b-a548-e2e9f0681994 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:18:04.506230 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b7adbe-3214-4a6b-a548-e2e9f0681994 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:18:04.506230 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b7adbe-3214-4a6b-a548-e2e9f0681994 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:18:04.506503 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc803c88-1dea-47de-bdb0-ac9c8a1e92a6 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:18:04.522389 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:18:04.554565 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d5b462-8c64-4ac4-bba4-8930927b3df7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:18:04.555687 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d5b462-8c64-4ac4-bba4-8930927b3df7 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:18:04.555974 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d5b462-8c64-4ac4-bba4-8930927b3df7 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:18:04.556289 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d5b462-8c64-4ac4-bba4-8930927b3df7 None admin] PATH_INFO: `/v3/services` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:18:04.557520 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d5b462-8c64-4ac4-bba4-8930927b3df7 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:18:04.557831 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d5b462-8c64-4ac4-bba4-8930927b3df7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:18:04.559965 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d5b462-8c64-4ac4-bba4-8930927b3df7 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:18:04.567098 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 422/824] 10.222.0.30 () {56 vars in 1124 bytes} [Tue Sep 6 04:18:04 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:18:04.567948 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d5b462-8c64-4ac4-bba4-8930927b3df7 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:18:04.589961 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b7adbe-3214-4a6b-a548-e2e9f0681994 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:18:04.591451 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b7adbe-3214-4a6b-a548-e2e9f0681994 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:18:04.695216 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d5b462-8c64-4ac4-bba4-8930927b3df7 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:18:04.717550 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 403/825] 10.222.0.30 () {58 vars in 1282 bytes} [Tue Sep 6 04:18:04 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 06 04:18:04.718327 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b7adbe-3214-4a6b-a548-e2e9f0681994 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:18:04.744623 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33503c51-d948-4170-ad0f-4befa5562b94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:18:04.745542 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33503c51-d948-4170-ad0f-4befa5562b94 None admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:18:04.745866 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33503c51-d948-4170-ad0f-4befa5562b94 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:18:04.746149 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33503c51-d948-4170-ad0f-4befa5562b94 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:18:04.747024 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33503c51-d948-4170-ad0f-4befa5562b94 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:18:04.747251 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:18:04.748707 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33503c51-d948-4170-ad0f-4befa5562b94 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:18:04.825237 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b7adbe-3214-4a6b-a548-e2e9f0681994 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:18:04.863658 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad88d387-26fc-4a6a-bb7c-4b8f85ca1ac2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:18:04.863868 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 423/826] 10.222.0.30 () {60 vars in 1144 bytes} [Tue Sep 6 04:18:04 2022] POST /identity/v3/endpoints => generated 337 bytes in 296 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 04:18:04.864374 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad88d387-26fc-4a6a-bb7c-4b8f85ca1ac2 None admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:18:04.864674 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad88d387-26fc-4a6a-bb7c-4b8f85ca1ac2 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:18:04.864836 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33503c51-d948-4170-ad0f-4befa5562b94 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:18:04.864937 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad88d387-26fc-4a6a-bb7c-4b8f85ca1ac2 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:18:04.865735 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad88d387-26fc-4a6a-bb7c-4b8f85ca1ac2 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:18:04.867179 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad88d387-26fc-4a6a-bb7c-4b8f85ca1ac2 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:18:04.915401 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:18:04.969247 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 404/827] 10.222.0.30 () {60 vars in 1144 bytes} [Tue Sep 6 04:18:04 2022] POST /identity/v3/endpoints => generated 346 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 04:18:05.013836 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33503c51-d948-4170-ad0f-4befa5562b94 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:18:05.069572 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7363d5d9-650b-40a9-983a-51e0663f662c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:18:05.070232 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7363d5d9-650b-40a9-983a-51e0663f662c None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:18:05.070513 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7363d5d9-650b-40a9-983a-51e0663f662c None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:18:05.070786 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7363d5d9-650b-40a9-983a-51e0663f662c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:18:05.083098 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7363d5d9-650b-40a9-983a-51e0663f662c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:18:05.084591 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7363d5d9-650b-40a9-983a-51e0663f662c None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:18:05.101189 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-7363d5d9-650b-40a9-983a-51e0663f662c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 04:18:05.101189 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 04:18:05.101189 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 04:18:05.101189 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 04:18:05.101189 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 04:18:05.101189 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 04:18:05.101189 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 04:18:05.101189 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 04:18:05.101189 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 04:18:05.101189 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 04:18:05.101189 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 04:18:05.101189 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 04:18:05.101189 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 04:18:05.101189 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 04:18:05.101189 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 04:18:05.101189 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 04:18:05.102268 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 04:18:05.102268 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 04:18:05.102268 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 04:18:05.102268 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 04:18:05.102268 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 04:18:05.102268 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 04:18:05.102268 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 04:18:05.102268 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 04:18:05.102268 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 04:18:05.102268 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 04:18:05.102268 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 04:18:05.102268 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 04:18:05.102268 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 04:18:05.102268 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 04:18:05.102268 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 04:18:05.102268 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 04:18:05.102268 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 04:18:05.103167 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 04:18:05.103167 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 04:18:05.103167 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 04:18:05.103765 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 424/828] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Sep 6 04:18:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 04:18:05.109355 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad88d387-26fc-4a6a-bb7c-4b8f85ca1ac2 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:18:05.201032 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:18:05.222316 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad88d387-26fc-4a6a-bb7c-4b8f85ca1ac2 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:18:05.261026 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f21f03-fe5f-4cef-b471-dfd99b3c0f66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:18:05.261698 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f21f03-fe5f-4cef-b471-dfd99b3c0f66 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:18:05.261982 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f21f03-fe5f-4cef-b471-dfd99b3c0f66 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:18:05.262258 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f21f03-fe5f-4cef-b471-dfd99b3c0f66 None admin] PATH_INFO: `/v3/domains` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:18:05.262916 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f21f03-fe5f-4cef-b471-dfd99b3c0f66 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:18:05.263220 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f21f03-fe5f-4cef-b471-dfd99b3c0f66 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:18:05.264593 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f21f03-fe5f-4cef-b471-dfd99b3c0f66 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:18:05.277978 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 405/829] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Sep 6 04:18:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:18:05.283396 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7363d5d9-650b-40a9-983a-51e0663f662c None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:18:05.387878 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:18:05.410704 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7363d5d9-650b-40a9-983a-51e0663f662c None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:18:05.446917 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a317cd5c-bb1b-41d5-a11e-efda36ae332e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:18:05.447986 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a317cd5c-bb1b-41d5-a11e-efda36ae332e None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:18:05.448319 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a317cd5c-bb1b-41d5-a11e-efda36ae332e None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:18:05.448599 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a317cd5c-bb1b-41d5-a11e-efda36ae332e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:18:05.456036 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a317cd5c-bb1b-41d5-a11e-efda36ae332e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:18:05.457698 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a317cd5c-bb1b-41d5-a11e-efda36ae332e None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:18:05.470152 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-a317cd5c-bb1b-41d5-a11e-efda36ae332e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 04:18:05.470152 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 04:18:05.470152 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 04:18:05.470152 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 04:18:05.470152 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 04:18:05.470152 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 04:18:05.470152 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 04:18:05.470152 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 04:18:05.470152 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 04:18:05.470152 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 04:18:05.470152 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 04:18:05.470152 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 04:18:05.470152 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 06 04:18:05.470152 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 06 04:18:05.470152 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 06 04:18:05.472373 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 06 04:18:05.472373 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 04:18:05.472373 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 04:18:05.472373 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 04:18:05.472373 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 04:18:05.472373 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 04:18:05.472373 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 04:18:05.472373 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 04:18:05.472373 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 04:18:05.472373 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 04:18:05.472373 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 04:18:05.472373 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 04:18:05.472373 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 04:18:05.472373 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 04:18:05.472373 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 04:18:05.472373 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 04:18:05.472373 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 04:18:05.474541 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 06 04:18:05.474541 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 06 04:18:05.474541 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 06 04:18:05.474541 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 06 04:18:05.474541 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 06 04:18:05.474541 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 06 04:18:05.474541 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 04:18:05.474541 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 04:18:05.474541 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 425/830] 10.222.0.30 () {56 vars in 1111 bytes} [Tue Sep 6 04:18:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 04:18:05.477808 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f21f03-fe5f-4cef-b471-dfd99b3c0f66 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:18:05.584864 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f21f03-fe5f-4cef-b471-dfd99b3c0f66 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:18:05.627350 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3724ae30-6026-4776-89a2-38e3c075357a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:18:05.628466 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3724ae30-6026-4776-89a2-38e3c075357a None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:18:05.628466 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3724ae30-6026-4776-89a2-38e3c075357a None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:18:05.628902 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3724ae30-6026-4776-89a2-38e3c075357a None admin] PATH_INFO: `/v3/projects` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:18:05.633346 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3724ae30-6026-4776-89a2-38e3c075357a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:18:05.633346 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3724ae30-6026-4776-89a2-38e3c075357a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:18:05.633346 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3724ae30-6026-4776-89a2-38e3c075357a None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:18:05.641027 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:18:05.645872 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 406/831] 10.222.0.30 () {56 vars in 1148 bytes} [Tue Sep 6 04:18:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:18:05.651420 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a317cd5c-bb1b-41d5-a11e-efda36ae332e None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:18:05.761470 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a317cd5c-bb1b-41d5-a11e-efda36ae332e None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:18:05.797430 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17cb90bc-5bf1-47a7-8696-c426f72c14d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:18:05.798051 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cb90bc-5bf1-47a7-8696-c426f72c14d6 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:18:05.798332 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cb90bc-5bf1-47a7-8696-c426f72c14d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:18:05.798602 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cb90bc-5bf1-47a7-8696-c426f72c14d6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:18:05.799967 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17cb90bc-5bf1-47a7-8696-c426f72c14d6 None admin] RBAC: Adding query filter params (role.id=ea4c814713b149108e488de855b025e4, scope.project.id=b707733ce65c42edbc1fd95ad0a1c06c, user.id=9db138b77afa4b068244b8ef715404fe) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:18:05.800369 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17cb90bc-5bf1-47a7-8696-c426f72c14d6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:18:05.801945 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17cb90bc-5bf1-47a7-8696-c426f72c14d6 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:18:05.808478 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 426/832] 10.222.0.30 () {56 vars in 1374 bytes} [Tue Sep 6 04:18:05 2022] GET /identity/v3/role_assignments?user.id=9db138b77afa4b068244b8ef715404fe&scope.project.id=b707733ce65c42edbc1fd95ad0a1c06c&role.id=ea4c814713b149108e488de855b025e4 => generated 608 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:18:05.828368 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db877d2d-0d02-42f9-8cc0-6d2bee2e5153 None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:18:05.828696 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db877d2d-0d02-42f9-8cc0-6d2bee2e5153 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:18:05.828916 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db877d2d-0d02-42f9-8cc0-6d2bee2e5153 None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:18:05.831189 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 407/833] 10.222.0.30 () {54 vars in 927 bytes} [Tue Sep 6 04:18:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 04:18:05.838220 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e43a2c3-72a2-4d8d-a7d9-072f3a73eeab None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:18:05.838554 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e43a2c3-72a2-4d8d-a7d9-072f3a73eeab None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:18:05.838554 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e43a2c3-72a2-4d8d-a7d9-072f3a73eeab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:18:05.845379 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:18:05.859604 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-6e43a2c3-72a2-4d8d-a7d9-072f3a73eeab None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:18:05.860835 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-6e43a2c3-72a2-4d8d-a7d9-072f3a73eeab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:18:05.903429 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d196800-56a9-45f0-9e5c-77a58bdd9f5f None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:18:05.903766 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d196800-56a9-45f0-9e5c-77a58bdd9f5f None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:18:05.903980 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d196800-56a9-45f0-9e5c-77a58bdd9f5f None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:18:05.910373 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 408/834] 10.222.0.30 () {54 vars in 927 bytes} [Tue Sep 6 04:18:05 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 04:18:05.911681 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcf1f5e-1dcf-4b0f-9a63-9aa5940ea987 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:18:05.911968 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcf1f5e-1dcf-4b0f-9a63-9aa5940ea987 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:18:05.912246 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcf1f5e-1dcf-4b0f-9a63-9aa5940ea987 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:18:05.936381 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-1fcf1f5e-1dcf-4b0f-9a63-9aa5940ea987 None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:18:05.938657 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-1fcf1f5e-1dcf-4b0f-9a63-9aa5940ea987 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:18:05.947346 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 427/835] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Sep 6 04:18:05 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 04:18:06.019101 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 409/836] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Sep 6 04:18:05 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 04:18:06.236302 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e43a2c3-72a2-4d8d-a7d9-072f3a73eeab None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:18:06.309388 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fcf1f5e-1dcf-4b0f-9a63-9aa5940ea987 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:18:06.354993 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e43a2c3-72a2-4d8d-a7d9-072f3a73eeab None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:18:06.404900 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:18:06.412543 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300730a6-b764-48e4-bc09-b5f935b1cc8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:18:06.420547 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300730a6-b764-48e4-bc09-b5f935b1cc8d None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:18:06.420547 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300730a6-b764-48e4-bc09-b5f935b1cc8d None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:18:06.420547 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300730a6-b764-48e4-bc09-b5f935b1cc8d None admin] PATH_INFO: `/v3/roles/service` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:18:06.429426 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300730a6-b764-48e4-bc09-b5f935b1cc8d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 04:18:06.437414 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-300730a6-b764-48e4-bc09-b5f935b1cc8d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 06 04:18:06.437414 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 04:18:06.437414 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 04:18:06.437414 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 04:18:06.437414 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 04:18:06.437414 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 04:18:06.437414 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 04:18:06.437414 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 04:18:06.437414 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 04:18:06.437414 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 04:18:06.437414 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 04:18:06.437414 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 04:18:06.437414 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 04:18:06.437414 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 04:18:06.437414 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 04:18:06.437414 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 04:18:06.438388 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 04:18:06.438388 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 04:18:06.438388 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 04:18:06.438388 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 04:18:06.438388 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 04:18:06.438388 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 04:18:06.438388 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 04:18:06.438388 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 04:18:06.438388 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 04:18:06.438388 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 04:18:06.438388 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 04:18:06.438388 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 04:18:06.438388 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 04:18:06.438388 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 04:18:06.438388 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 04:18:06.438388 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 04:18:06.439251 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 04:18:06.439251 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 04:18:06.439251 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 04:18:06.439251 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 04:18:06.439251 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 04:18:06.439251 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 04:18:06.439251 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 04:18:06.439251 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 04:18:06.439251 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 04:18:06.439251 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 04:18:06.439251 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 06 04:18:06.439251 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 04:18:06.439251 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 428/837] 10.222.0.30 () {56 vars in 1102 bytes} [Tue Sep 6 04:18:06 2022] GET /identity/v3/roles/service => generated 85 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 04:18:06.445469 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300730a6-b764-48e4-bc09-b5f935b1cc8d None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:18:06.462581 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fcf1f5e-1dcf-4b0f-9a63-9aa5940ea987 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:18:06.495240 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:18:06.502249 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114af45e-e45e-4cda-b40d-eea7b66dac1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:18:06.503518 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114af45e-e45e-4cda-b40d-eea7b66dac1f None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:18:06.503803 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114af45e-e45e-4cda-b40d-eea7b66dac1f None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:18:06.504081 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114af45e-e45e-4cda-b40d-eea7b66dac1f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:18:06.516581 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114af45e-e45e-4cda-b40d-eea7b66dac1f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 04:18:06.522430 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-114af45e-e45e-4cda-b40d-eea7b66dac1f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 04:18:06.522430 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 04:18:06.522430 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 04:18:06.522430 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 04:18:06.522430 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 04:18:06.522430 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 04:18:06.522430 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 04:18:06.522430 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 04:18:06.522430 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 04:18:06.522430 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 04:18:06.522430 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 04:18:06.522430 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 04:18:06.522430 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 04:18:06.522430 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 04:18:06.522430 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 04:18:06.522430 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 04:18:06.523378 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 04:18:06.523378 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 04:18:06.523378 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 04:18:06.523378 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 04:18:06.523378 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 04:18:06.523378 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 04:18:06.523378 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 04:18:06.523378 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 04:18:06.523378 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 04:18:06.523378 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 04:18:06.523378 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 04:18:06.523378 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 04:18:06.523378 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 04:18:06.523378 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 04:18:06.523378 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 04:18:06.523378 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 04:18:06.524223 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 04:18:06.524223 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 04:18:06.524223 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 04:18:06.524223 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 04:18:06.524223 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 04:18:06.524223 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 04:18:06.524223 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 04:18:06.524223 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 04:18:06.524223 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 04:18:06.524223 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 04:18:06.524223 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 04:18:06.524223 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 04:18:06.524223 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 410/838] 10.222.0.30 () {56 vars in 1096 bytes} [Tue Sep 6 04:18:06 2022] GET /identity/v3/roles/admin => generated 83 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 04:18:06.531408 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114af45e-e45e-4cda-b40d-eea7b66dac1f None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:18:06.557756 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300730a6-b764-48e4-bc09-b5f935b1cc8d None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:18:06.593172 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c10c70f-e4f3-423a-99f0-94ac5919bb59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:18:06.594085 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c10c70f-e4f3-423a-99f0-94ac5919bb59 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:18:06.594542 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c10c70f-e4f3-423a-99f0-94ac5919bb59 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:18:06.594989 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c10c70f-e4f3-423a-99f0-94ac5919bb59 None admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:18:06.595867 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c10c70f-e4f3-423a-99f0-94ac5919bb59 None admin] RBAC: Adding query filter params (name=service) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:18:06.596458 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c10c70f-e4f3-423a-99f0-94ac5919bb59 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:18:06.598061 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c10c70f-e4f3-423a-99f0-94ac5919bb59 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:18:06.606989 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 429/839] 10.222.0.30 () {56 vars in 1103 bytes} [Tue Sep 6 04:18:06 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:18:06.613577 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c10c70f-e4f3-423a-99f0-94ac5919bb59 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:18:06.650889 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114af45e-e45e-4cda-b40d-eea7b66dac1f None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:18:06.673666 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:18:06.687179 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b32f51a-8a27-4d79-ac9c-07080c098e91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:18:06.687794 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b32f51a-8a27-4d79-ac9c-07080c098e91 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:18:06.688090 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b32f51a-8a27-4d79-ac9c-07080c098e91 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:18:06.688403 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b32f51a-8a27-4d79-ac9c-07080c098e91 None admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:18:06.689016 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b32f51a-8a27-4d79-ac9c-07080c098e91 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:18:06.689341 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b32f51a-8a27-4d79-ac9c-07080c098e91 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:18:06.691878 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b32f51a-8a27-4d79-ac9c-07080c098e91 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:18:06.700899 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 411/840] 10.222.0.30 () {56 vars in 1099 bytes} [Tue Sep 6 04:18:06 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:18:06.710727 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b32f51a-8a27-4d79-ac9c-07080c098e91 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:18:06.722329 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c10c70f-e4f3-423a-99f0-94ac5919bb59 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:18:06.774615 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de35a462-c604-46a1-b63a-4f918e4dc3e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:18:06.776648 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de35a462-c604-46a1-b63a-4f918e4dc3e2 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:18:06.777257 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de35a462-c604-46a1-b63a-4f918e4dc3e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:18:06.778182 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de35a462-c604-46a1-b63a-4f918e4dc3e2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:18:06.787080 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:18:06.803723 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de35a462-c604-46a1-b63a-4f918e4dc3e2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:18:06.807183 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de35a462-c604-46a1-b63a-4f918e4dc3e2 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:18:06.824071 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-de35a462-c604-46a1-b63a-4f918e4dc3e2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 04:18:06.824071 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 04:18:06.824071 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 04:18:06.824071 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 04:18:06.824071 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 04:18:06.824071 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 04:18:06.824071 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 04:18:06.824071 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 04:18:06.824071 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 04:18:06.824071 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 04:18:06.824071 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 04:18:06.824071 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 04:18:06.824071 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 04:18:06.824071 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 04:18:06.824071 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 04:18:06.824071 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 04:18:06.825369 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 04:18:06.825369 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 04:18:06.825369 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 04:18:06.825369 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 04:18:06.825369 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 04:18:06.825369 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 04:18:06.825369 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 04:18:06.825369 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 04:18:06.825369 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 04:18:06.825369 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 04:18:06.825369 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 04:18:06.825369 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 04:18:06.825369 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 04:18:06.825369 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 04:18:06.825369 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 04:18:06.825369 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 04:18:06.825369 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 04:18:06.826472 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 04:18:06.826472 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 04:18:06.826472 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 04:18:06.827772 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 430/841] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Sep 6 04:18:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 04:18:06.834570 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de35a462-c604-46a1-b63a-4f918e4dc3e2 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:18:06.845291 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b32f51a-8a27-4d79-ac9c-07080c098e91 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:18:06.884724 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812e4c76-fbee-47b5-a235-72dede7307fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:18:06.885515 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812e4c76-fbee-47b5-a235-72dede7307fe None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:18:06.885863 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812e4c76-fbee-47b5-a235-72dede7307fe None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:18:06.886203 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812e4c76-fbee-47b5-a235-72dede7307fe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:18:06.899132 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812e4c76-fbee-47b5-a235-72dede7307fe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:18:06.900851 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812e4c76-fbee-47b5-a235-72dede7307fe None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:18:06.917234 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-812e4c76-fbee-47b5-a235-72dede7307fe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 04:18:06.917234 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 04:18:06.917234 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 04:18:06.917234 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 04:18:06.917234 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 04:18:06.917234 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 04:18:06.917234 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 04:18:06.917234 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 04:18:06.917234 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 04:18:06.917234 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 04:18:06.917234 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 04:18:06.917234 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 04:18:06.917234 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 04:18:06.917234 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 04:18:06.917234 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 04:18:06.917234 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 04:18:06.918410 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 04:18:06.918410 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 04:18:06.918410 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 04:18:06.918410 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 04:18:06.918410 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 04:18:06.918410 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 04:18:06.918410 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 04:18:06.918410 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 04:18:06.918410 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 04:18:06.918410 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 04:18:06.918410 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 04:18:06.918410 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 04:18:06.918410 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 04:18:06.918410 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 04:18:06.918410 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 04:18:06.918410 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 04:18:06.918410 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 04:18:06.919411 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 04:18:06.919411 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 04:18:06.919411 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 04:18:06.923618 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 412/842] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Sep 6 04:18:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 04:18:06.925854 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812e4c76-fbee-47b5-a235-72dede7307fe None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:18:06.966414 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de35a462-c604-46a1-b63a-4f918e4dc3e2 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:18:06.977045 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:18:07.013340 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4191c0-80d8-4dfc-a951-a9c3b3bc8cf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:18:07.013940 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4191c0-80d8-4dfc-a951-a9c3b3bc8cf9 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:18:07.014265 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4191c0-80d8-4dfc-a951-a9c3b3bc8cf9 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:18:07.014675 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4191c0-80d8-4dfc-a951-a9c3b3bc8cf9 None admin] PATH_INFO: `/v3/domains` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:18:07.015688 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4191c0-80d8-4dfc-a951-a9c3b3bc8cf9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:18:07.016310 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4191c0-80d8-4dfc-a951-a9c3b3bc8cf9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:18:07.018275 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4191c0-80d8-4dfc-a951-a9c3b3bc8cf9 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:18:07.034955 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 431/843] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Sep 6 04:18:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:18:07.037533 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4191c0-80d8-4dfc-a951-a9c3b3bc8cf9 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:18:07.057024 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812e4c76-fbee-47b5-a235-72dede7307fe None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:18:07.060399 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:18:07.101030 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129bc0d6-1102-4167-bb7b-1539e4dde4d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:18:07.102038 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129bc0d6-1102-4167-bb7b-1539e4dde4d9 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:18:07.102510 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129bc0d6-1102-4167-bb7b-1539e4dde4d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:18:07.103026 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129bc0d6-1102-4167-bb7b-1539e4dde4d9 None admin] PATH_INFO: `/v3/domains` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:18:07.103829 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-129bc0d6-1102-4167-bb7b-1539e4dde4d9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:18:07.104286 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-129bc0d6-1102-4167-bb7b-1539e4dde4d9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:18:07.106064 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-129bc0d6-1102-4167-bb7b-1539e4dde4d9 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:18:07.125253 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 413/844] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Sep 6 04:18:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:18:07.126572 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129bc0d6-1102-4167-bb7b-1539e4dde4d9 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:18:07.152412 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4191c0-80d8-4dfc-a951-a9c3b3bc8cf9 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:18:07.189540 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade0fe8f-eb60-45c2-bd1d-e88df6db9d63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:18:07.190827 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade0fe8f-eb60-45c2-bd1d-e88df6db9d63 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:18:07.191286 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade0fe8f-eb60-45c2-bd1d-e88df6db9d63 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:18:07.191575 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade0fe8f-eb60-45c2-bd1d-e88df6db9d63 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:18:07.199813 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade0fe8f-eb60-45c2-bd1d-e88df6db9d63 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:18:07.201420 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade0fe8f-eb60-45c2-bd1d-e88df6db9d63 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:18:07.213872 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-ade0fe8f-eb60-45c2-bd1d-e88df6db9d63 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 06 04:18:07.213872 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 04:18:07.213872 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 04:18:07.213872 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 04:18:07.213872 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 04:18:07.213872 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 04:18:07.213872 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 04:18:07.213872 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 04:18:07.213872 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 04:18:07.213872 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 04:18:07.213872 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 04:18:07.213872 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 04:18:07.213872 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 06 04:18:07.213872 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 06 04:18:07.213872 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 06 04:18:07.213872 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 06 04:18:07.214859 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 04:18:07.214859 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 04:18:07.214859 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 06 04:18:07.214859 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 04:18:07.214859 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 06 04:18:07.214859 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 04:18:07.214859 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 04:18:07.214859 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 04:18:07.214859 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 04:18:07.214859 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 04:18:07.214859 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 04:18:07.214859 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 04:18:07.214859 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 04:18:07.214859 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 04:18:07.214859 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 04:18:07.214859 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 04:18:07.214859 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 04:18:07.214859 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 04:18:07.216351 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 04:18:07.216351 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 04:18:07.216351 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 06 04:18:07.216351 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 06 04:18:07.216351 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 06 04:18:07.216351 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 06 04:18:07.216351 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 06 04:18:07.216351 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 06 04:18:07.216351 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 06 04:18:07.216351 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 04:18:07.216351 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 432/845] 10.222.0.30 () {56 vars in 1117 bytes} [Tue Sep 6 04:18:07 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 04:18:07.216956 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcde2072-1413-41ba-ad26-9258c40cdc11 None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:18:07.216956 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcde2072-1413-41ba-ad26-9258c40cdc11 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:18:07.216956 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcde2072-1413-41ba-ad26-9258c40cdc11 None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:18:07.219288 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 433/846] 10.222.0.30 () {54 vars in 927 bytes} [Tue Sep 6 04:18:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 04:18:07.220356 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcde2072-1413-41ba-ad26-9258c40cdc11 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:18:07.248065 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129bc0d6-1102-4167-bb7b-1539e4dde4d9 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:18:07.249641 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:18:07.286425 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07580715-5184-45d7-a774-bff1f7f7f226 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:18:07.287064 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07580715-5184-45d7-a774-bff1f7f7f226 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:18:07.287342 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07580715-5184-45d7-a774-bff1f7f7f226 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:18:07.287610 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07580715-5184-45d7-a774-bff1f7f7f226 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:18:07.294696 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07580715-5184-45d7-a774-bff1f7f7f226 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:18:07.296131 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07580715-5184-45d7-a774-bff1f7f7f226 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:18:07.308861 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-07580715-5184-45d7-a774-bff1f7f7f226 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 06 04:18:07.308861 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 04:18:07.308861 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 04:18:07.308861 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 04:18:07.308861 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 04:18:07.308861 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 04:18:07.308861 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 04:18:07.308861 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 04:18:07.308861 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 04:18:07.308861 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 04:18:07.308861 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 04:18:07.308861 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 04:18:07.308861 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 06 04:18:07.308861 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 06 04:18:07.308861 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 06 04:18:07.308861 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 06 04:18:07.310240 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 04:18:07.310240 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 04:18:07.310240 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 06 04:18:07.310240 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 04:18:07.310240 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 06 04:18:07.310240 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 04:18:07.310240 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 04:18:07.310240 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 04:18:07.310240 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 04:18:07.310240 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 04:18:07.310240 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 04:18:07.310240 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 04:18:07.310240 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 04:18:07.310240 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 04:18:07.310240 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 04:18:07.310240 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 04:18:07.310240 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 04:18:07.310240 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 04:18:07.311423 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 04:18:07.311423 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 04:18:07.311423 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 06 04:18:07.311423 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 06 04:18:07.311423 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 06 04:18:07.311423 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 06 04:18:07.311423 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 06 04:18:07.311423 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 06 04:18:07.311423 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 06 04:18:07.311423 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 04:18:07.314796 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 414/847] 10.222.0.30 () {56 vars in 1096 bytes} [Tue Sep 6 04:18:07 2022] GET /identity/v3/users/swift => generated 83 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 04:18:07.316375 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ef2a57-1ef7-467a-b561-57b05e0056a8 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:18:07.316661 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ef2a57-1ef7-467a-b561-57b05e0056a8 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:18:07.316936 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ef2a57-1ef7-467a-b561-57b05e0056a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:18:07.333788 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcde2072-1413-41ba-ad26-9258c40cdc11 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:18:07.336076 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-11ef2a57-1ef7-467a-b561-57b05e0056a8 None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:18:07.337273 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-11ef2a57-1ef7-467a-b561-57b05e0056a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:18:07.355998 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:18:07.376199 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afd55b3-8a43-462b-8b23-0f2aa65b081f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:18:07.377130 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afd55b3-8a43-462b-8b23-0f2aa65b081f None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:18:07.377443 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afd55b3-8a43-462b-8b23-0f2aa65b081f None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:18:07.377718 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afd55b3-8a43-462b-8b23-0f2aa65b081f None admin] PATH_INFO: `/v3/users` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:18:07.378460 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8afd55b3-8a43-462b-8b23-0f2aa65b081f None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:18:07.378764 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8afd55b3-8a43-462b-8b23-0f2aa65b081f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:18:07.382192 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8afd55b3-8a43-462b-8b23-0f2aa65b081f None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:18:07.409617 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 434/848] 10.222.0.30 () {56 vars in 1149 bytes} [Tue Sep 6 04:18:07 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 04:18:07.410706 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afd55b3-8a43-462b-8b23-0f2aa65b081f None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:18:07.415088 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 415/849] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Sep 6 04:18:07 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 04:18:07.415501 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ef2a57-1ef7-467a-b561-57b05e0056a8 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:18:07.506452 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:18:07.523421 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afd55b3-8a43-462b-8b23-0f2aa65b081f None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:18:07.529650 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ef2a57-1ef7-467a-b561-57b05e0056a8 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:18:07.561207 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d478b59-6070-4882-8952-50fefeaa8a4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:18:07.562197 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d478b59-6070-4882-8952-50fefeaa8a4c None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:18:07.562678 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d478b59-6070-4882-8952-50fefeaa8a4c None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:18:07.563174 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d478b59-6070-4882-8952-50fefeaa8a4c None admin] PATH_INFO: `/v3/users` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:18:07.564179 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d478b59-6070-4882-8952-50fefeaa8a4c None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:18:07.564846 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d478b59-6070-4882-8952-50fefeaa8a4c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:18:07.566899 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d478b59-6070-4882-8952-50fefeaa8a4c None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:18:07.568245 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3c5f47-1df2-4930-9f35-68f912223b73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:18:07.568859 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3c5f47-1df2-4930-9f35-68f912223b73 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:18:07.569140 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3c5f47-1df2-4930-9f35-68f912223b73 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:18:07.569444 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3c5f47-1df2-4930-9f35-68f912223b73 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:18:07.591600 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 435/850] 10.222.0.30 () {56 vars in 1135 bytes} [Tue Sep 6 04:18:07 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:18:07.592799 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d478b59-6070-4882-8952-50fefeaa8a4c None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:18:07.597028 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:18:07.646705 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b3c5f47-1df2-4930-9f35-68f912223b73 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:18:07.648190 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b3c5f47-1df2-4930-9f35-68f912223b73 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:18:07.699831 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d478b59-6070-4882-8952-50fefeaa8a4c None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:18:07.738191 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81101aa-a631-4729-b6ef-3b494a2f0b43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:18:07.738814 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81101aa-a631-4729-b6ef-3b494a2f0b43 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:18:07.739094 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81101aa-a631-4729-b6ef-3b494a2f0b43 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:18:07.739531 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81101aa-a631-4729-b6ef-3b494a2f0b43 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:18:07.757391 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:18:07.763760 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 416/851] 10.222.0.30 () {58 vars in 1282 bytes} [Tue Sep 6 04:18:07 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 06 04:18:07.763760 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3c5f47-1df2-4930-9f35-68f912223b73 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:18:07.820417 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:18:07.823691 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d81101aa-a631-4729-b6ef-3b494a2f0b43 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:18:07.825381 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d81101aa-a631-4729-b6ef-3b494a2f0b43 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:18:07.871259 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3c5f47-1df2-4930-9f35-68f912223b73 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:18:07.906067 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64681c47-2eba-4c47-8786-7dc81f0fe05f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:18:07.906635 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64681c47-2eba-4c47-8786-7dc81f0fe05f None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:18:07.906903 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64681c47-2eba-4c47-8786-7dc81f0fe05f None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:18:07.907179 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64681c47-2eba-4c47-8786-7dc81f0fe05f None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:18:07.907758 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64681c47-2eba-4c47-8786-7dc81f0fe05f None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:18:07.909711 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64681c47-2eba-4c47-8786-7dc81f0fe05f None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:18:07.918394 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-64681c47-2eba-4c47-8786-7dc81f0fe05f None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 06 04:18:07.918394 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 04:18:07.918394 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 06 04:18:07.918394 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 06 04:18:07.918394 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 06 04:18:07.918394 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 06 04:18:07.918394 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 06 04:18:07.918394 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 04:18:07.918394 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 06 04:18:07.918394 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 04:18:07.918394 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 06 04:18:07.918394 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 04:18:07.918394 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 04:18:07.918394 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 04:18:07.918394 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 04:18:07.918394 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 04:18:07.918394 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 04:18:07.919410 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 04:18:07.919410 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 04:18:07.919410 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 04:18:07.919410 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 04:18:07.919410 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 04:18:07.919410 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 04:18:07.919410 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 06 04:18:07.919410 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 06 04:18:07.919410 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 06 04:18:07.919410 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 06 04:18:07.919410 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 04:18:07.919410 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 04:18:07.919410 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 04:18:07.919410 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 04:18:07.919410 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 04:18:07.919410 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 04:18:07.920298 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 04:18:07.920298 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 04:18:07.920298 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 04:18:07.920298 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 04:18:07.920298 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 04:18:07.920298 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 04:18:07.920298 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 04:18:07.920298 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 04:18:07.920298 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 04:18:07.920298 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 04:18:07.920298 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 06 04:18:07.920298 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 04:18:07.920298 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 06 04:18:07.920298 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 04:18:07.920298 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 417/852] 10.222.0.30 () {56 vars in 1114 bytes} [Tue Sep 6 04:18:07 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 04:18:07.921105 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64681c47-2eba-4c47-8786-7dc81f0fe05f None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:18:07.939293 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 436/853] 10.222.0.30 () {58 vars in 1282 bytes} [Tue Sep 6 04:18:07 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 06 04:18:07.940801 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81101aa-a631-4729-b6ef-3b494a2f0b43 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:18:07.984141 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:18:08.026976 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64681c47-2eba-4c47-8786-7dc81f0fe05f None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:18:08.040967 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81101aa-a631-4729-b6ef-3b494a2f0b43 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:18:08.043526 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:18:08.061752 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181b1678-c68f-4b95-97fa-9dfc593480a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:18:08.062385 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181b1678-c68f-4b95-97fa-9dfc593480a9 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:18:08.062665 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181b1678-c68f-4b95-97fa-9dfc593480a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:18:08.062936 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181b1678-c68f-4b95-97fa-9dfc593480a9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:18:08.073816 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-181b1678-c68f-4b95-97fa-9dfc593480a9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:18:08.075125 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe14034a-0ce3-4a1c-8e40-2c10f44075ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:18:08.075727 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe14034a-0ce3-4a1c-8e40-2c10f44075ed None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:18:08.075819 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-181b1678-c68f-4b95-97fa-9dfc593480a9 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:18:08.076002 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe14034a-0ce3-4a1c-8e40-2c10f44075ed None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:18:08.076269 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe14034a-0ce3-4a1c-8e40-2c10f44075ed None admin] PATH_INFO: `/v3/services` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:18:08.076888 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe14034a-0ce3-4a1c-8e40-2c10f44075ed None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:18:08.077183 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe14034a-0ce3-4a1c-8e40-2c10f44075ed None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:18:08.078558 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe14034a-0ce3-4a1c-8e40-2c10f44075ed None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:18:08.083095 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 437/854] 10.222.0.30 () {56 vars in 1114 bytes} [Tue Sep 6 04:18:07 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:18:08.083861 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe14034a-0ce3-4a1c-8e40-2c10f44075ed None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:18:08.090181 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-181b1678-c68f-4b95-97fa-9dfc593480a9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 04:18:08.090181 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 04:18:08.090181 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 04:18:08.090181 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 04:18:08.090181 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 04:18:08.090181 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 04:18:08.090181 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 04:18:08.090181 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 04:18:08.090181 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 04:18:08.090181 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 04:18:08.090181 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 04:18:08.090181 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 04:18:08.090181 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 04:18:08.090181 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 04:18:08.090181 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 04:18:08.090181 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 04:18:08.091318 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 04:18:08.091318 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 04:18:08.091318 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 04:18:08.091318 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 04:18:08.091318 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 04:18:08.091318 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 04:18:08.091318 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 04:18:08.091318 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 04:18:08.091318 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 04:18:08.091318 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 04:18:08.091318 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 04:18:08.091318 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 04:18:08.091318 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 04:18:08.091318 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 04:18:08.091318 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 04:18:08.091318 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 04:18:08.091318 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 04:18:08.092576 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 04:18:08.092576 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 04:18:08.092576 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 04:18:08.092576 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 418/855] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Sep 6 04:18:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 04:18:08.092862 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4f9c96-7a4b-4836-bb3a-9638aeca2ba0 None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:18:08.092951 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4f9c96-7a4b-4836-bb3a-9638aeca2ba0 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:18:08.093215 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4f9c96-7a4b-4836-bb3a-9638aeca2ba0 None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:18:08.094611 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 419/856] 10.222.0.30 () {54 vars in 927 bytes} [Tue Sep 6 04:18:08 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 04:18:08.099242 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4f9c96-7a4b-4836-bb3a-9638aeca2ba0 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:18:08.183689 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe14034a-0ce3-4a1c-8e40-2c10f44075ed None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:18:08.191058 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:18:08.200046 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4f9c96-7a4b-4836-bb3a-9638aeca2ba0 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:18:08.217147 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de749c0-848a-4d54-adaf-248bddf7be82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:18:08.217802 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de749c0-848a-4d54-adaf-248bddf7be82 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:18:08.218081 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de749c0-848a-4d54-adaf-248bddf7be82 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:18:08.218354 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de749c0-848a-4d54-adaf-248bddf7be82 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:18:08.228307 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de749c0-848a-4d54-adaf-248bddf7be82 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:18:08.229786 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de749c0-848a-4d54-adaf-248bddf7be82 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:18:08.234060 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1c8b46-2d51-42b4-940c-6fa007b48df5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:18:08.234663 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1c8b46-2d51-42b4-940c-6fa007b48df5 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:18:08.234941 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1c8b46-2d51-42b4-940c-6fa007b48df5 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:18:08.235213 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1c8b46-2d51-42b4-940c-6fa007b48df5 None admin] PATH_INFO: `/v3/services` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:18:08.235839 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1c8b46-2d51-42b4-940c-6fa007b48df5 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:18:08.236141 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1c8b46-2d51-42b4-940c-6fa007b48df5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:18:08.238390 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1c8b46-2d51-42b4-940c-6fa007b48df5 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:18:08.242767 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 420/857] 10.222.0.30 () {56 vars in 1114 bytes} [Tue Sep 6 04:18:08 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:18:08.243534 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1c8b46-2d51-42b4-940c-6fa007b48df5 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:18:08.244773 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-4de749c0-848a-4d54-adaf-248bddf7be82 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 04:18:08.244773 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 04:18:08.244773 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 04:18:08.244773 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 04:18:08.244773 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 04:18:08.244773 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 04:18:08.244773 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 04:18:08.244773 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 04:18:08.244773 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 04:18:08.244773 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 04:18:08.244773 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 04:18:08.244773 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 04:18:08.244773 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 04:18:08.244773 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 04:18:08.244773 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 04:18:08.244773 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 04:18:08.245895 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 04:18:08.245895 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 04:18:08.245895 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 04:18:08.245895 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 04:18:08.245895 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 04:18:08.245895 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 04:18:08.245895 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 04:18:08.245895 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 04:18:08.245895 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 04:18:08.245895 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 04:18:08.245895 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 04:18:08.245895 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 04:18:08.245895 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 04:18:08.245895 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 04:18:08.245895 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 04:18:08.245895 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 04:18:08.245895 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 04:18:08.246780 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 04:18:08.246780 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 04:18:08.246780 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 04:18:08.246780 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 438/858] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Sep 6 04:18:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 04:18:08.247181 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cff70b7-2a8b-4b89-8208-87216136c4d2 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:18:08.247510 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cff70b7-2a8b-4b89-8208-87216136c4d2 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:18:08.247636 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cff70b7-2a8b-4b89-8208-87216136c4d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:18:08.268586 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-9cff70b7-2a8b-4b89-8208-87216136c4d2 None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:18:08.269883 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-9cff70b7-2a8b-4b89-8208-87216136c4d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:18:08.293174 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:18:08.347575 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 439/859] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Sep 6 04:18:08 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 04:18:08.357540 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cff70b7-2a8b-4b89-8208-87216136c4d2 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:18:08.386685 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1c8b46-2d51-42b4-940c-6fa007b48df5 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:18:08.429251 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59b6d5f-d108-41e5-ab5e-572fdd2baf12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:18:08.430323 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59b6d5f-d108-41e5-ab5e-572fdd2baf12 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:18:08.430808 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59b6d5f-d108-41e5-ab5e-572fdd2baf12 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:18:08.431278 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59b6d5f-d108-41e5-ab5e-572fdd2baf12 None admin] PATH_INFO: `/v3/domains` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:18:08.432129 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59b6d5f-d108-41e5-ab5e-572fdd2baf12 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:18:08.432659 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59b6d5f-d108-41e5-ab5e-572fdd2baf12 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:18:08.434319 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59b6d5f-d108-41e5-ab5e-572fdd2baf12 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:18:08.452631 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 421/860] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Sep 6 04:18:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:18:08.454000 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59b6d5f-d108-41e5-ab5e-572fdd2baf12 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:18:08.472623 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:18:08.489612 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cff70b7-2a8b-4b89-8208-87216136c4d2 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:18:08.529856 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fbda11-d5fc-4379-b320-2ef3bef5ca2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:18:08.530530 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fbda11-d5fc-4379-b320-2ef3bef5ca2c None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:18:08.530814 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fbda11-d5fc-4379-b320-2ef3bef5ca2c None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:18:08.531085 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fbda11-d5fc-4379-b320-2ef3bef5ca2c None admin] PATH_INFO: `/v3/domains` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:18:08.531733 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72fbda11-d5fc-4379-b320-2ef3bef5ca2c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:18:08.532033 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72fbda11-d5fc-4379-b320-2ef3bef5ca2c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:18:08.533427 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72fbda11-d5fc-4379-b320-2ef3bef5ca2c None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:18:08.550573 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 440/861] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Sep 6 04:18:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:18:08.552132 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fbda11-d5fc-4379-b320-2ef3bef5ca2c None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:18:08.562446 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59b6d5f-d108-41e5-ab5e-572fdd2baf12 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:18:08.565640 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:18:08.601814 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0d4685-dc81-439f-9b2b-04b2e13c0a1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:18:08.602617 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0d4685-dc81-439f-9b2b-04b2e13c0a1f None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:18:08.603001 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0d4685-dc81-439f-9b2b-04b2e13c0a1f None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:18:08.603374 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0d4685-dc81-439f-9b2b-04b2e13c0a1f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:18:08.610586 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0d4685-dc81-439f-9b2b-04b2e13c0a1f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:18:08.612772 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0d4685-dc81-439f-9b2b-04b2e13c0a1f None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:18:08.624450 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-0f0d4685-dc81-439f-9b2b-04b2e13c0a1f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 04:18:08.624450 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 04:18:08.624450 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 04:18:08.624450 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 04:18:08.624450 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 04:18:08.624450 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 04:18:08.624450 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 04:18:08.624450 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 04:18:08.624450 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 04:18:08.624450 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 04:18:08.624450 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 04:18:08.624450 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 04:18:08.624450 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 06 04:18:08.624450 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 06 04:18:08.624450 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 06 04:18:08.625550 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 06 04:18:08.625550 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 04:18:08.625550 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 04:18:08.625550 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 04:18:08.625550 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 04:18:08.625550 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 04:18:08.625550 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 04:18:08.625550 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 04:18:08.625550 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 04:18:08.625550 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 04:18:08.625550 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 04:18:08.625550 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 04:18:08.625550 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 04:18:08.625550 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 04:18:08.625550 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 04:18:08.625550 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 04:18:08.625550 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 04:18:08.626657 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 06 04:18:08.626657 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 06 04:18:08.626657 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 06 04:18:08.626657 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 06 04:18:08.626657 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 06 04:18:08.626657 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 06 04:18:08.626657 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 04:18:08.626657 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 04:18:08.629468 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 422/862] 10.222.0.30 () {56 vars in 1111 bytes} [Tue Sep 6 04:18:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 04:18:08.630369 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0d4685-dc81-439f-9b2b-04b2e13c0a1f None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:18:08.668707 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fbda11-d5fc-4379-b320-2ef3bef5ca2c None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:18:08.705650 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc998ff-1ceb-47d8-b9de-d2c39ce83163 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:18:08.706550 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc998ff-1ceb-47d8-b9de-d2c39ce83163 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:18:08.706836 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc998ff-1ceb-47d8-b9de-d2c39ce83163 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:18:08.707108 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc998ff-1ceb-47d8-b9de-d2c39ce83163 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:18:08.721614 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc998ff-1ceb-47d8-b9de-d2c39ce83163 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:18:08.721614 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc998ff-1ceb-47d8-b9de-d2c39ce83163 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:18:08.727551 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-6fc998ff-1ceb-47d8-b9de-d2c39ce83163 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 04:18:08.727551 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 04:18:08.727551 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 04:18:08.727551 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 04:18:08.727551 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 04:18:08.727551 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 04:18:08.727551 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 04:18:08.727551 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 04:18:08.727551 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 04:18:08.727551 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 04:18:08.727551 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 04:18:08.727551 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 04:18:08.727551 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 06 04:18:08.727551 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 06 04:18:08.727551 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 06 04:18:08.728594 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 06 04:18:08.728594 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 04:18:08.728594 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 04:18:08.728594 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 04:18:08.728594 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 04:18:08.728594 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 04:18:08.728594 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 04:18:08.728594 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 04:18:08.728594 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 04:18:08.728594 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 04:18:08.728594 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 04:18:08.728594 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 04:18:08.728594 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 04:18:08.728594 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 04:18:08.728594 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 04:18:08.728594 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 04:18:08.728594 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 04:18:08.729557 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 06 04:18:08.729557 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 06 04:18:08.729557 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 06 04:18:08.729557 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 06 04:18:08.729557 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 06 04:18:08.729557 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 06 04:18:08.729557 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 04:18:08.729557 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 04:18:08.729557 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 441/863] 10.222.0.30 () {56 vars in 1111 bytes} [Tue Sep 6 04:18:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 04:18:08.730046 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc998ff-1ceb-47d8-b9de-d2c39ce83163 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:18:08.745394 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:18:08.746436 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0d4685-dc81-439f-9b2b-04b2e13c0a1f None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:18:08.783246 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42d447b-507d-4f75-9e77-3cf45f2d6e9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:18:08.785116 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42d447b-507d-4f75-9e77-3cf45f2d6e9d None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:18:08.785822 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42d447b-507d-4f75-9e77-3cf45f2d6e9d None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:18:08.786667 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42d447b-507d-4f75-9e77-3cf45f2d6e9d None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:18:08.787288 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42d447b-507d-4f75-9e77-3cf45f2d6e9d None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:18:08.788666 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42d447b-507d-4f75-9e77-3cf45f2d6e9d None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:18:08.798118 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-d42d447b-507d-4f75-9e77-3cf45f2d6e9d None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 06 04:18:08.798118 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 04:18:08.798118 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 06 04:18:08.798118 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 06 04:18:08.798118 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 06 04:18:08.798118 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 06 04:18:08.798118 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 06 04:18:08.798118 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 04:18:08.798118 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 06 04:18:08.798118 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 04:18:08.798118 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 06 04:18:08.798118 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 04:18:08.798118 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 04:18:08.798118 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 04:18:08.798118 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 04:18:08.798118 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 04:18:08.799056 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 04:18:08.799056 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 04:18:08.799056 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 04:18:08.799056 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 04:18:08.799056 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 04:18:08.799056 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 04:18:08.799056 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 04:18:08.799056 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 06 04:18:08.799056 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 06 04:18:08.799056 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 06 04:18:08.799056 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 06 04:18:08.799056 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 04:18:08.799056 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 04:18:08.799056 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 04:18:08.799056 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 04:18:08.799056 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 04:18:08.799056 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 04:18:08.800606 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 04:18:08.800606 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 04:18:08.800606 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 04:18:08.800606 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 04:18:08.800606 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 04:18:08.800606 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 04:18:08.800606 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 04:18:08.800606 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 04:18:08.800606 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 04:18:08.800606 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 04:18:08.800606 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 06 04:18:08.800606 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 04:18:08.800606 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 06 04:18:08.800606 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 04:18:08.800606 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 423/864] 10.222.0.30 () {56 vars in 1132 bytes} [Tue Sep 6 04:18:08 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 04:18:08.802218 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42d447b-507d-4f75-9e77-3cf45f2d6e9d None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:18:08.822564 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:18:08.845441 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc998ff-1ceb-47d8-b9de-d2c39ce83163 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:18:08.881579 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c619ebc-6ae4-4035-b641-5854700298fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:18:08.882332 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c619ebc-6ae4-4035-b641-5854700298fa None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:18:08.882713 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c619ebc-6ae4-4035-b641-5854700298fa None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:18:08.883101 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c619ebc-6ae4-4035-b641-5854700298fa None admin] PATH_INFO: `/v3/projects` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:18:08.883878 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c619ebc-6ae4-4035-b641-5854700298fa None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:18:08.884301 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c619ebc-6ae4-4035-b641-5854700298fa None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:18:08.885862 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c619ebc-6ae4-4035-b641-5854700298fa None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:18:08.903700 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 442/865] 10.222.0.30 () {56 vars in 1148 bytes} [Tue Sep 6 04:18:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:18:08.903700 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c619ebc-6ae4-4035-b641-5854700298fa None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:18:08.915270 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42d447b-507d-4f75-9e77-3cf45f2d6e9d None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:18:08.950101 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ecbea32-0ceb-4da7-8a8d-6ba1e25d3ee4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:18:08.950807 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecbea32-0ceb-4da7-8a8d-6ba1e25d3ee4 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:18:08.951159 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecbea32-0ceb-4da7-8a8d-6ba1e25d3ee4 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:18:08.951494 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecbea32-0ceb-4da7-8a8d-6ba1e25d3ee4 None admin] PATH_INFO: `/v3/projects` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:18:08.952254 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ecbea32-0ceb-4da7-8a8d-6ba1e25d3ee4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:18:08.952626 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ecbea32-0ceb-4da7-8a8d-6ba1e25d3ee4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:18:08.954162 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ecbea32-0ceb-4da7-8a8d-6ba1e25d3ee4 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:18:08.971579 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 424/866] 10.222.0.30 () {56 vars in 1148 bytes} [Tue Sep 6 04:18:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:18:08.971579 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ecbea32-0ceb-4da7-8a8d-6ba1e25d3ee4 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:18:08.978336 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:18:09.015543 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c619ebc-6ae4-4035-b641-5854700298fa None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:18:09.051581 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:18:09.052938 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3b7fed-b1e3-4c41-90f6-1da7cf91c91e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:18:09.053594 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3b7fed-b1e3-4c41-90f6-1da7cf91c91e None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:18:09.053871 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3b7fed-b1e3-4c41-90f6-1da7cf91c91e None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:18:09.054146 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3b7fed-b1e3-4c41-90f6-1da7cf91c91e None admin] PATH_INFO: `/v3/services` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:18:09.054932 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb3b7fed-b1e3-4c41-90f6-1da7cf91c91e None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:18:09.055235 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb3b7fed-b1e3-4c41-90f6-1da7cf91c91e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:18:09.056624 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb3b7fed-b1e3-4c41-90f6-1da7cf91c91e None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:18:09.061286 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 443/867] 10.222.0.30 () {56 vars in 1126 bytes} [Tue Sep 6 04:18:08 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:18:09.062143 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3b7fed-b1e3-4c41-90f6-1da7cf91c91e None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:18:09.079260 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ecbea32-0ceb-4da7-8a8d-6ba1e25d3ee4 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:18:09.115519 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c5c1d6-9233-4a36-895c-35d9df77e435 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:18:09.116137 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c5c1d6-9233-4a36-895c-35d9df77e435 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:18:09.116418 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c5c1d6-9233-4a36-895c-35d9df77e435 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:18:09.116692 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c5c1d6-9233-4a36-895c-35d9df77e435 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:18:09.117686 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c5c1d6-9233-4a36-895c-35d9df77e435 None admin] RBAC: Adding query filter params (role.id=83cf0a194f8d45e792411e2d9afdfe50, scope.project.id=b707733ce65c42edbc1fd95ad0a1c06c, user.id=da5fed17101a4935bcc38b72caf9d263) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:18:09.117993 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c5c1d6-9233-4a36-895c-35d9df77e435 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:18:09.119802 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c5c1d6-9233-4a36-895c-35d9df77e435 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:18:09.125590 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 425/868] 10.222.0.30 () {56 vars in 1374 bytes} [Tue Sep 6 04:18:08 2022] GET /identity/v3/role_assignments?user.id=da5fed17101a4935bcc38b72caf9d263&scope.project.id=b707733ce65c42edbc1fd95ad0a1c06c&role.id=83cf0a194f8d45e792411e2d9afdfe50 => generated 608 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:18:09.126369 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c5c1d6-9233-4a36-895c-35d9df77e435 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:18:09.190670 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3b7fed-b1e3-4c41-90f6-1da7cf91c91e None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:18:09.229661 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:18:09.245948 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faff025a-2d6b-4079-9a36-494c6b0a648c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:18:09.247306 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faff025a-2d6b-4079-9a36-494c6b0a648c None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:18:09.247735 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faff025a-2d6b-4079-9a36-494c6b0a648c None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:18:09.248151 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faff025a-2d6b-4079-9a36-494c6b0a648c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:18:09.249115 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faff025a-2d6b-4079-9a36-494c6b0a648c None admin] RBAC: Adding query filter params (role.id=ea4c814713b149108e488de855b025e4, scope.project.id=b707733ce65c42edbc1fd95ad0a1c06c, user.id=a676ec7f92a144edbeca8e6029ba9ee6) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:18:09.250246 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faff025a-2d6b-4079-9a36-494c6b0a648c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:18:09.253024 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faff025a-2d6b-4079-9a36-494c6b0a648c None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:18:09.269337 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c5c1d6-9233-4a36-895c-35d9df77e435 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:18:09.269744 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 444/869] 10.222.0.30 () {56 vars in 1374 bytes} [Tue Sep 6 04:18:09 2022] GET /identity/v3/role_assignments?user.id=a676ec7f92a144edbeca8e6029ba9ee6&scope.project.id=b707733ce65c42edbc1fd95ad0a1c06c&role.id=ea4c814713b149108e488de855b025e4 => generated 608 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:18:09.312271 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01502526-1c22-42f6-a036-f0408651f965 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:18:09.312904 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01502526-1c22-42f6-a036-f0408651f965 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:18:09.313184 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01502526-1c22-42f6-a036-f0408651f965 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:18:09.313491 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01502526-1c22-42f6-a036-f0408651f965 None admin] PATH_INFO: `/v3/services` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:18:09.314150 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01502526-1c22-42f6-a036-f0408651f965 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:18:09.314455 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01502526-1c22-42f6-a036-f0408651f965 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:18:09.316425 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01502526-1c22-42f6-a036-f0408651f965 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:18:09.322303 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 426/870] 10.222.0.30 () {56 vars in 1126 bytes} [Tue Sep 6 04:18:09 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:18:10.234142 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8c2343-8cb6-4130-9a6a-f1130793ef60 None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:18:10.234651 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8c2343-8cb6-4130-9a6a-f1130793ef60 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:18:10.234758 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8c2343-8cb6-4130-9a6a-f1130793ef60 None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:18:10.236346 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 445/871] 10.222.0.30 () {54 vars in 927 bytes} [Tue Sep 6 04:18:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 04:18:10.241837 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733d40ef-6789-4476-a44c-69b33ef3b5a8 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:18:10.242254 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733d40ef-6789-4476-a44c-69b33ef3b5a8 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:18:10.242421 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733d40ef-6789-4476-a44c-69b33ef3b5a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:18:10.256097 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:18:10.261762 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-733d40ef-6789-4476-a44c-69b33ef3b5a8 None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:18:10.262962 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-733d40ef-6789-4476-a44c-69b33ef3b5a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:18:10.345459 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 427/872] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Sep 6 04:18:10 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 04:18:10.624362 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8c2343-8cb6-4130-9a6a-f1130793ef60 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:18:10.738666 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8c2343-8cb6-4130-9a6a-f1130793ef60 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:18:10.774398 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6669be-f4e5-4ec8-94c0-65f96bdd2bbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:18:10.775054 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6669be-f4e5-4ec8-94c0-65f96bdd2bbc None admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:18:10.775330 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6669be-f4e5-4ec8-94c0-65f96bdd2bbc None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:18:10.775654 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6669be-f4e5-4ec8-94c0-65f96bdd2bbc None admin] PATH_INFO: `/v3/services` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:18:10.776327 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6669be-f4e5-4ec8-94c0-65f96bdd2bbc None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:18:10.777780 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6669be-f4e5-4ec8-94c0-65f96bdd2bbc None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:18:10.881293 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 446/873] 10.222.0.30 () {60 vars in 1141 bytes} [Tue Sep 6 04:18:10 2022] POST /identity/v3/services => generated 251 bytes in 257 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 04:18:11.206806 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fb4fa4-a91d-4899-a4dd-b876064ba642 None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:18:11.207224 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fb4fa4-a91d-4899-a4dd-b876064ba642 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:18:11.207602 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fb4fa4-a91d-4899-a4dd-b876064ba642 None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:18:11.209290 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 428/874] 10.222.0.30 () {54 vars in 927 bytes} [Tue Sep 6 04:18:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 04:18:11.214542 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe481c6-41b9-40f4-8b5e-d9815c9340e3 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:18:11.214935 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe481c6-41b9-40f4-8b5e-d9815c9340e3 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:18:11.215330 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe481c6-41b9-40f4-8b5e-d9815c9340e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:18:11.234978 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-ebe481c6-41b9-40f4-8b5e-d9815c9340e3 None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:18:11.236175 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-ebe481c6-41b9-40f4-8b5e-d9815c9340e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:18:11.275474 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7b17de-9a0e-4c64-9433-d5df31715ac5 None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:18:11.275977 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7b17de-9a0e-4c64-9433-d5df31715ac5 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:18:11.276052 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7b17de-9a0e-4c64-9433-d5df31715ac5 None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:18:11.277619 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 429/875] 10.222.0.30 () {54 vars in 927 bytes} [Tue Sep 6 04:18:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 06 04:18:11.282518 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9250d9e4-cdab-479c-b34c-91ad3c08d0e3 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:18:11.282803 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9250d9e4-cdab-479c-b34c-91ad3c08d0e3 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:18:11.283082 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9250d9e4-cdab-479c-b34c-91ad3c08d0e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:18:11.290528 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:18:11.304409 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-9250d9e4-cdab-479c-b34c-91ad3c08d0e3 None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:18:11.316396 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-9250d9e4-cdab-479c-b34c-91ad3c08d0e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:18:11.316518 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 447/876] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Sep 6 04:18:11 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 04:18:11.316518 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2839f1-4678-4223-a07b-bca5555a0817 None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:18:11.316518 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2839f1-4678-4223-a07b-bca5555a0817 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:18:11.316518 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2839f1-4678-4223-a07b-bca5555a0817 None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:18:11.317426 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 448/877] 10.222.0.30 () {54 vars in 927 bytes} [Tue Sep 6 04:18:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 04:18:11.326049 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fcb707-37d9-4342-a586-3f49e6f8cd5d None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:18:11.326678 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fcb707-37d9-4342-a586-3f49e6f8cd5d None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:18:11.327170 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fcb707-37d9-4342-a586-3f49e6f8cd5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:18:11.347012 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-e8fcb707-37d9-4342-a586-3f49e6f8cd5d None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:18:11.348789 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-e8fcb707-37d9-4342-a586-3f49e6f8cd5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:18:11.389371 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 430/878] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Sep 6 04:18:11 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 06 04:18:11.430908 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 449/879] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Sep 6 04:18:11 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 06 04:18:11.590474 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9250d9e4-cdab-479c-b34c-91ad3c08d0e3 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:18:11.658303 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:18:11.677052 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8fcb707-37d9-4342-a586-3f49e6f8cd5d None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:18:11.700683 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9250d9e4-cdab-479c-b34c-91ad3c08d0e3 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:18:11.736147 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2989753-8bf8-433d-8166-fff1e47e7336 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:18:11.737730 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2989753-8bf8-433d-8166-fff1e47e7336 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:18:11.737730 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2989753-8bf8-433d-8166-fff1e47e7336 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:18:11.738022 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2989753-8bf8-433d-8166-fff1e47e7336 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:18:11.749376 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2989753-8bf8-433d-8166-fff1e47e7336 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 04:18:11.754488 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-d2989753-8bf8-433d-8166-fff1e47e7336 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 06 04:18:11.754488 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 04:18:11.754488 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 04:18:11.754488 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 04:18:11.754488 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 04:18:11.754488 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 04:18:11.754488 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 04:18:11.754488 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 04:18:11.754488 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 04:18:11.754488 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 04:18:11.754488 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 04:18:11.754488 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 04:18:11.754488 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 04:18:11.754488 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 04:18:11.754488 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 04:18:11.754488 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 04:18:11.755606 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 04:18:11.755606 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 04:18:11.755606 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 04:18:11.755606 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 04:18:11.755606 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 04:18:11.755606 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 04:18:11.755606 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 04:18:11.755606 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 04:18:11.755606 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 04:18:11.755606 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 04:18:11.755606 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 04:18:11.755606 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 04:18:11.755606 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 04:18:11.755606 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 04:18:11.755606 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 04:18:11.755606 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 04:18:11.756453 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 04:18:11.756453 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 04:18:11.756453 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 04:18:11.756453 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 04:18:11.756453 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 04:18:11.756453 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 04:18:11.756453 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 04:18:11.756453 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 04:18:11.756453 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 04:18:11.756453 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 04:18:11.756453 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 06 04:18:11.756453 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 04:18:11.756453 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 431/880] 10.222.0.30 () {56 vars in 1120 bytes} [Tue Sep 6 04:18:11 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 04:18:11.757185 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2989753-8bf8-433d-8166-fff1e47e7336 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:18:11.805137 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8fcb707-37d9-4342-a586-3f49e6f8cd5d None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:18:11.820539 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:18:11.844551 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331c01ac-c664-4fe6-a32b-dac794e3dedc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:18:11.845089 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331c01ac-c664-4fe6-a32b-dac794e3dedc None admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:18:11.845496 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331c01ac-c664-4fe6-a32b-dac794e3dedc None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:18:11.845774 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331c01ac-c664-4fe6-a32b-dac794e3dedc None admin] PATH_INFO: `/v3/services` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:18:11.847827 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331c01ac-c664-4fe6-a32b-dac794e3dedc None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:18:11.849412 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331c01ac-c664-4fe6-a32b-dac794e3dedc None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:18:11.865188 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2989753-8bf8-433d-8166-fff1e47e7336 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:18:11.897512 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:18:11.906085 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bacf66fb-5706-497c-9c1c-08abc42d84d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:18:11.906725 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacf66fb-5706-497c-9c1c-08abc42d84d7 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:18:11.907007 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacf66fb-5706-497c-9c1c-08abc42d84d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:18:11.907282 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacf66fb-5706-497c-9c1c-08abc42d84d7 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:18:11.907873 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bacf66fb-5706-497c-9c1c-08abc42d84d7 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:18:11.909875 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bacf66fb-5706-497c-9c1c-08abc42d84d7 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:18:11.918507 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-bacf66fb-5706-497c-9c1c-08abc42d84d7 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 06 04:18:11.918507 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 04:18:11.918507 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 06 04:18:11.918507 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 06 04:18:11.918507 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 06 04:18:11.918507 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 06 04:18:11.918507 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 06 04:18:11.918507 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 04:18:11.918507 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 06 04:18:11.918507 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 04:18:11.918507 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 06 04:18:11.918507 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 04:18:11.918507 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 04:18:11.918507 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 04:18:11.918507 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 04:18:11.918507 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 04:18:11.918507 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 04:18:11.922176 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 04:18:11.922176 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 04:18:11.922176 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 04:18:11.922176 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 04:18:11.922176 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 04:18:11.922176 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 04:18:11.922176 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 06 04:18:11.922176 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 06 04:18:11.922176 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 06 04:18:11.922176 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 06 04:18:11.922176 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 04:18:11.922176 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 04:18:11.922176 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 04:18:11.922176 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 04:18:11.922176 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 04:18:11.922176 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 04:18:11.923012 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 04:18:11.923012 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 04:18:11.923012 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 04:18:11.923012 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 04:18:11.923012 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 04:18:11.923012 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 04:18:11.923012 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 04:18:11.923012 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 04:18:11.923012 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 04:18:11.923012 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 04:18:11.923012 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 06 04:18:11.923012 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 04:18:11.923012 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 06 04:18:11.923012 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 04:18:11.923012 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 432/881] 10.222.0.30 () {56 vars in 1126 bytes} [Tue Sep 6 04:18:11 2022] GET /identity/v3/services/object-store => generated 93 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 04:18:11.923854 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bacf66fb-5706-497c-9c1c-08abc42d84d7 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:18:11.965755 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 450/882] 10.222.0.30 () {60 vars in 1141 bytes} [Tue Sep 6 04:18:11 2022] POST /identity/v3/services => generated 269 bytes in 285 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 04:18:11.965755 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331c01ac-c664-4fe6-a32b-dac794e3dedc None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:18:12.056247 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bacf66fb-5706-497c-9c1c-08abc42d84d7 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:18:12.108614 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee9cebb-2a28-4ea0-9864-206a423bb631 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:18:12.109165 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee9cebb-2a28-4ea0-9864-206a423bb631 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:18:12.109474 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee9cebb-2a28-4ea0-9864-206a423bb631 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:18:12.109742 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee9cebb-2a28-4ea0-9864-206a423bb631 None admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:18:12.110403 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee9cebb-2a28-4ea0-9864-206a423bb631 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:18:12.110703 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee9cebb-2a28-4ea0-9864-206a423bb631 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:18:12.111267 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331c01ac-c664-4fe6-a32b-dac794e3dedc None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:18:12.113420 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee9cebb-2a28-4ea0-9864-206a423bb631 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:18:12.122028 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 433/883] 10.222.0.30 () {56 vars in 1115 bytes} [Tue Sep 6 04:18:11 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:18:12.128368 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee9cebb-2a28-4ea0-9864-206a423bb631 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:18:12.160086 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e4f0ba-8c86-4d44-a865-7c77181bf388 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:18:12.161434 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e4f0ba-8c86-4d44-a865-7c77181bf388 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:18:12.161911 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e4f0ba-8c86-4d44-a865-7c77181bf388 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:18:12.162812 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e4f0ba-8c86-4d44-a865-7c77181bf388 None admin] PATH_INFO: `/v3/services` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:18:12.163692 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e4f0ba-8c86-4d44-a865-7c77181bf388 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:18:12.164001 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e4f0ba-8c86-4d44-a865-7c77181bf388 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:18:12.166090 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e4f0ba-8c86-4d44-a865-7c77181bf388 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:18:12.171169 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 451/884] 10.222.0.30 () {56 vars in 1122 bytes} [Tue Sep 6 04:18:11 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:18:12.178388 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e4f0ba-8c86-4d44-a865-7c77181bf388 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:18:12.213436 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:18:12.218370 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:18:12.286146 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee9cebb-2a28-4ea0-9864-206a423bb631 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:18:12.287751 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e4f0ba-8c86-4d44-a865-7c77181bf388 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:18:12.326281 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8d0d5c-1fd8-4224-b1f1-958e00255c10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:18:12.327069 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8d0d5c-1fd8-4224-b1f1-958e00255c10 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:18:12.327442 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8d0d5c-1fd8-4224-b1f1-958e00255c10 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:18:12.327809 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8d0d5c-1fd8-4224-b1f1-958e00255c10 None admin] PATH_INFO: `/v3/services` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:18:12.328697 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8d0d5c-1fd8-4224-b1f1-958e00255c10 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:18:12.328925 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8d0d5c-1fd8-4224-b1f1-958e00255c10 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:18:12.330515 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8d0d5c-1fd8-4224-b1f1-958e00255c10 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:18:12.335154 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1efd113-e679-47cc-9617-fbd6cb113e11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:18:12.335991 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 452/885] 10.222.0.30 () {56 vars in 1122 bytes} [Tue Sep 6 04:18:12 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:18:12.336111 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1efd113-e679-47cc-9617-fbd6cb113e11 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:18:12.336522 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1efd113-e679-47cc-9617-fbd6cb113e11 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:18:12.336863 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1efd113-e679-47cc-9617-fbd6cb113e11 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:18:12.350750 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1efd113-e679-47cc-9617-fbd6cb113e11 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:18:12.352249 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1efd113-e679-47cc-9617-fbd6cb113e11 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:18:12.369420 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-d1efd113-e679-47cc-9617-fbd6cb113e11 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 04:18:12.369420 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 04:18:12.369420 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 04:18:12.369420 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 04:18:12.369420 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 04:18:12.369420 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 04:18:12.369420 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 04:18:12.369420 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 04:18:12.369420 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 04:18:12.369420 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 04:18:12.369420 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 04:18:12.369420 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 04:18:12.369420 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 04:18:12.369420 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 04:18:12.369420 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 04:18:12.369420 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 04:18:12.370493 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 04:18:12.370493 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 04:18:12.370493 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 04:18:12.370493 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 04:18:12.370493 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 04:18:12.370493 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 04:18:12.370493 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 04:18:12.370493 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 04:18:12.370493 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 04:18:12.370493 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 04:18:12.370493 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 04:18:12.370493 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 04:18:12.370493 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 04:18:12.370493 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 04:18:12.370493 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 04:18:12.370493 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 04:18:12.370493 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 04:18:12.371402 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 04:18:12.371402 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 04:18:12.371402 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 04:18:12.371402 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 434/886] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Sep 6 04:18:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 04:18:12.376626 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8d0d5c-1fd8-4224-b1f1-958e00255c10 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:18:12.499011 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:18:12.502922 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8d0d5c-1fd8-4224-b1f1-958e00255c10 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:18:12.538287 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22740a45-ea98-44e6-bc11-3964524c3c23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:18:12.538898 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22740a45-ea98-44e6-bc11-3964524c3c23 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:18:12.539175 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22740a45-ea98-44e6-bc11-3964524c3c23 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:18:12.539473 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22740a45-ea98-44e6-bc11-3964524c3c23 None admin] PATH_INFO: `/v3/domains` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:18:12.540117 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22740a45-ea98-44e6-bc11-3964524c3c23 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:18:12.540416 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22740a45-ea98-44e6-bc11-3964524c3c23 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:18:12.541821 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22740a45-ea98-44e6-bc11-3964524c3c23 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:18:12.554541 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 453/887] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Sep 6 04:18:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:18:12.560155 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1efd113-e679-47cc-9617-fbd6cb113e11 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:18:12.670572 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:18:12.674642 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1efd113-e679-47cc-9617-fbd6cb113e11 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:18:12.710083 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20cc0e02-7cff-465b-90bc-fbf2507b8a99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:18:12.711024 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cc0e02-7cff-465b-90bc-fbf2507b8a99 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:18:12.711484 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cc0e02-7cff-465b-90bc-fbf2507b8a99 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:18:12.711937 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cc0e02-7cff-465b-90bc-fbf2507b8a99 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:18:12.720750 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20cc0e02-7cff-465b-90bc-fbf2507b8a99 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:18:12.722785 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20cc0e02-7cff-465b-90bc-fbf2507b8a99 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:18:12.734995 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-20cc0e02-7cff-465b-90bc-fbf2507b8a99 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 06 04:18:12.734995 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 04:18:12.734995 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 04:18:12.734995 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 04:18:12.734995 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 04:18:12.734995 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 04:18:12.734995 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 04:18:12.734995 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 04:18:12.734995 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 04:18:12.734995 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 04:18:12.734995 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 04:18:12.734995 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 04:18:12.734995 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 06 04:18:12.734995 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 06 04:18:12.734995 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 06 04:18:12.734995 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 06 04:18:12.735947 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 04:18:12.735947 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 04:18:12.735947 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 06 04:18:12.735947 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 04:18:12.735947 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 06 04:18:12.735947 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 04:18:12.735947 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 04:18:12.735947 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 04:18:12.735947 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 04:18:12.735947 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 04:18:12.735947 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 04:18:12.735947 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 04:18:12.735947 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 04:18:12.735947 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 04:18:12.735947 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 04:18:12.735947 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 04:18:12.735947 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 04:18:12.735947 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 04:18:12.737462 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 04:18:12.737462 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 04:18:12.737462 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 06 04:18:12.737462 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 06 04:18:12.737462 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 06 04:18:12.737462 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 06 04:18:12.737462 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 06 04:18:12.737462 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 06 04:18:12.737462 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 06 04:18:12.737462 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 04:18:12.739866 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 435/888] 10.222.0.30 () {56 vars in 1117 bytes} [Tue Sep 6 04:18:12 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 04:18:12.746410 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22740a45-ea98-44e6-bc11-3964524c3c23 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:18:12.854361 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22740a45-ea98-44e6-bc11-3964524c3c23 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:18:12.883427 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4601e0b-b178-40cf-aa75-f013ebd5c0bc None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:18:12.883868 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4601e0b-b178-40cf-aa75-f013ebd5c0bc None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:18:12.884250 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4601e0b-b178-40cf-aa75-f013ebd5c0bc None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:18:12.887031 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 436/889] 10.222.0.30 () {54 vars in 927 bytes} [Tue Sep 6 04:18:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 06 04:18:12.891117 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933ce8bc-6481-4834-ae0b-b5fda64ac78c None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:18:12.891267 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca9a024-27e9-496d-bec5-254d78e877d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:18:12.891458 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933ce8bc-6481-4834-ae0b-b5fda64ac78c None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:18:12.891675 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933ce8bc-6481-4834-ae0b-b5fda64ac78c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:18:12.891769 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca9a024-27e9-496d-bec5-254d78e877d8 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:18:12.892042 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca9a024-27e9-496d-bec5-254d78e877d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:18:12.892314 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca9a024-27e9-496d-bec5-254d78e877d8 None admin] PATH_INFO: `/v3/users` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:18:12.893084 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca9a024-27e9-496d-bec5-254d78e877d8 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:18:12.896611 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca9a024-27e9-496d-bec5-254d78e877d8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:18:12.896611 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca9a024-27e9-496d-bec5-254d78e877d8 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:18:12.914416 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 454/890] 10.222.0.30 () {56 vars in 1149 bytes} [Tue Sep 6 04:18:12 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:18:12.921390 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-933ce8bc-6481-4834-ae0b-b5fda64ac78c None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:18:12.921390 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-933ce8bc-6481-4834-ae0b-b5fda64ac78c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:18:12.921614 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca9a024-27e9-496d-bec5-254d78e877d8 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:18:12.940941 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:18:12.996977 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 437/891] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Sep 6 04:18:12 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 06 04:18:13.036926 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca9a024-27e9-496d-bec5-254d78e877d8 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:18:13.073983 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df48bf8c-a3b4-4bc7-bebc-fc72e8295486 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:18:13.074841 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df48bf8c-a3b4-4bc7-bebc-fc72e8295486 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:18:13.075267 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df48bf8c-a3b4-4bc7-bebc-fc72e8295486 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:18:13.075685 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df48bf8c-a3b4-4bc7-bebc-fc72e8295486 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:18:13.152079 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df48bf8c-a3b4-4bc7-bebc-fc72e8295486 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:18:13.153807 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df48bf8c-a3b4-4bc7-bebc-fc72e8295486 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:18:13.173612 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:18:13.262005 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 455/892] 10.222.0.30 () {58 vars in 1282 bytes} [Tue Sep 6 04:18:12 2022] GET /identity/v3/auth/tokens => generated 1372 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 06 04:18:13.270402 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933ce8bc-6481-4834-ae0b-b5fda64ac78c None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:18:13.293280 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df48bf8c-a3b4-4bc7-bebc-fc72e8295486 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:18:13.421632 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:18:13.423275 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df48bf8c-a3b4-4bc7-bebc-fc72e8295486 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:18:13.435010 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933ce8bc-6481-4834-ae0b-b5fda64ac78c None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:18:13.481675 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:18:13.484154 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b042c5d5-1dd4-4ae2-ae35-ec05dbdfb96e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:18:13.485128 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b042c5d5-1dd4-4ae2-ae35-ec05dbdfb96e None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:18:13.485650 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b042c5d5-1dd4-4ae2-ae35-ec05dbdfb96e None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:18:13.486128 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b042c5d5-1dd4-4ae2-ae35-ec05dbdfb96e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:18:13.498985 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b042c5d5-1dd4-4ae2-ae35-ec05dbdfb96e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:18:13.501569 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b042c5d5-1dd4-4ae2-ae35-ec05dbdfb96e None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:18:13.511760 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7780ac-e865-4e04-b7a8-24e9c38481cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:18:13.512416 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7780ac-e865-4e04-b7a8-24e9c38481cc None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:18:13.512698 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7780ac-e865-4e04-b7a8-24e9c38481cc None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:18:13.512969 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7780ac-e865-4e04-b7a8-24e9c38481cc None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:18:13.513608 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d7780ac-e865-4e04-b7a8-24e9c38481cc None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:18:13.519797 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d7780ac-e865-4e04-b7a8-24e9c38481cc None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:18:13.524210 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-b042c5d5-1dd4-4ae2-ae35-ec05dbdfb96e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 04:18:13.524210 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 04:18:13.524210 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 04:18:13.524210 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 04:18:13.524210 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 04:18:13.524210 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 04:18:13.524210 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 04:18:13.524210 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 04:18:13.524210 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 04:18:13.524210 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 04:18:13.524210 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 04:18:13.524210 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 04:18:13.524210 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 04:18:13.524210 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 04:18:13.524210 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 04:18:13.524210 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 04:18:13.525139 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 04:18:13.525139 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 04:18:13.525139 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 04:18:13.525139 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 04:18:13.525139 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 04:18:13.525139 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 04:18:13.525139 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 04:18:13.525139 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 04:18:13.525139 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 04:18:13.525139 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 04:18:13.525139 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 04:18:13.525139 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 04:18:13.525139 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 04:18:13.525139 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 04:18:13.525139 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 04:18:13.525139 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 04:18:13.525139 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 04:18:13.526067 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 04:18:13.526067 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 04:18:13.526067 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 04:18:13.528569 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 438/893] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Sep 6 04:18:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 04:18:13.534645 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-6d7780ac-e865-4e04-b7a8-24e9c38481cc None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 06 04:18:13.534645 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 04:18:13.534645 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 06 04:18:13.534645 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 06 04:18:13.534645 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 06 04:18:13.534645 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 06 04:18:13.534645 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 06 04:18:13.534645 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 04:18:13.534645 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 06 04:18:13.534645 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 04:18:13.534645 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 06 04:18:13.534645 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 04:18:13.534645 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 04:18:13.534645 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 04:18:13.534645 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 04:18:13.534645 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 04:18:13.534645 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 04:18:13.535626 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 04:18:13.535626 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 04:18:13.535626 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 04:18:13.535626 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 04:18:13.535626 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 04:18:13.535626 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 04:18:13.535626 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 06 04:18:13.535626 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 06 04:18:13.535626 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 06 04:18:13.535626 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 06 04:18:13.535626 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 04:18:13.535626 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 04:18:13.535626 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 04:18:13.535626 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 04:18:13.535626 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 04:18:13.535626 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 04:18:13.536828 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 04:18:13.536828 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 04:18:13.536828 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 04:18:13.536828 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 04:18:13.536828 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 04:18:13.536828 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 04:18:13.536828 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 04:18:13.536828 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 04:18:13.536828 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 04:18:13.536828 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 04:18:13.536828 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 06 04:18:13.536828 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 04:18:13.536828 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 06 04:18:13.536828 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 04:18:13.538594 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b042c5d5-1dd4-4ae2-ae35-ec05dbdfb96e None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:18:13.549534 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 456/894] 10.222.0.30 () {56 vars in 1114 bytes} [Tue Sep 6 04:18:13 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 04:18:13.550491 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7780ac-e865-4e04-b7a8-24e9c38481cc None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:18:13.665939 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b042c5d5-1dd4-4ae2-ae35-ec05dbdfb96e None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:18:13.711044 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a7d4c6-cd93-43e1-bfd0-616b7cfbace8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:18:13.712666 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a7d4c6-cd93-43e1-bfd0-616b7cfbace8 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:18:13.713171 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a7d4c6-cd93-43e1-bfd0-616b7cfbace8 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:18:13.713681 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a7d4c6-cd93-43e1-bfd0-616b7cfbace8 None admin] PATH_INFO: `/v3/domains` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:18:13.714884 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a7d4c6-cd93-43e1-bfd0-616b7cfbace8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:18:13.715418 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a7d4c6-cd93-43e1-bfd0-616b7cfbace8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:18:13.717963 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a7d4c6-cd93-43e1-bfd0-616b7cfbace8 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:18:13.734955 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 439/895] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Sep 6 04:18:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:18:13.740794 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a7d4c6-cd93-43e1-bfd0-616b7cfbace8 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:18:13.749926 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:18:13.787548 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7780ac-e865-4e04-b7a8-24e9c38481cc None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:18:13.865579 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ed6a05-39ba-4fd5-bc7d-b9ff4296d960 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:18:13.866212 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ed6a05-39ba-4fd5-bc7d-b9ff4296d960 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:18:13.866493 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ed6a05-39ba-4fd5-bc7d-b9ff4296d960 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:18:13.866768 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ed6a05-39ba-4fd5-bc7d-b9ff4296d960 None admin] PATH_INFO: `/v3/services` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:18:13.869319 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ed6a05-39ba-4fd5-bc7d-b9ff4296d960 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:18:13.869633 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ed6a05-39ba-4fd5-bc7d-b9ff4296d960 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:18:13.873696 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ed6a05-39ba-4fd5-bc7d-b9ff4296d960 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:18:13.895033 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 457/896] 10.222.0.30 () {56 vars in 1114 bytes} [Tue Sep 6 04:18:13 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 345 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:18:13.895984 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ed6a05-39ba-4fd5-bc7d-b9ff4296d960 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:18:13.902520 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a7d4c6-cd93-43e1-bfd0-616b7cfbace8 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:18:13.948128 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b78b433-e4d6-4315-a54d-2ff2d5d6932b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:18:13.949746 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b78b433-e4d6-4315-a54d-2ff2d5d6932b None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:18:13.950256 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b78b433-e4d6-4315-a54d-2ff2d5d6932b None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:18:13.950736 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b78b433-e4d6-4315-a54d-2ff2d5d6932b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:18:13.961140 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b78b433-e4d6-4315-a54d-2ff2d5d6932b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:18:13.963531 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b78b433-e4d6-4315-a54d-2ff2d5d6932b None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:18:13.966013 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:18:13.981268 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-4b78b433-e4d6-4315-a54d-2ff2d5d6932b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 04:18:13.981268 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 04:18:13.981268 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 04:18:13.981268 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 04:18:13.981268 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 04:18:13.981268 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 04:18:13.981268 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 04:18:13.981268 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 04:18:13.981268 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 04:18:13.981268 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 04:18:13.981268 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 04:18:13.981268 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 04:18:13.981268 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 06 04:18:13.981268 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 06 04:18:13.981268 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 06 04:18:13.982827 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 06 04:18:13.982827 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 04:18:13.982827 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 04:18:13.982827 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 04:18:13.982827 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 04:18:13.982827 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 04:18:13.982827 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 04:18:13.982827 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 04:18:13.982827 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 04:18:13.982827 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 04:18:13.982827 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 04:18:13.982827 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 04:18:13.982827 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 04:18:13.982827 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 04:18:13.982827 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 04:18:13.982827 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 04:18:13.982827 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 04:18:13.983735 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 06 04:18:13.983735 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 06 04:18:13.983735 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 06 04:18:13.983735 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 06 04:18:13.983735 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 06 04:18:13.983735 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 06 04:18:13.983735 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 04:18:13.983735 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 04:18:13.986357 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 440/897] 10.222.0.30 () {56 vars in 1111 bytes} [Tue Sep 6 04:18:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 04:18:13.992812 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b78b433-e4d6-4315-a54d-2ff2d5d6932b None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:18:14.027502 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ed6a05-39ba-4fd5-bc7d-b9ff4296d960 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:18:14.061586 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:18:14.067957 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c877e242-dd4a-4905-9a14-068e18169b5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:18:14.076818 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c877e242-dd4a-4905-9a14-068e18169b5f None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:18:14.076818 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c877e242-dd4a-4905-9a14-068e18169b5f None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:18:14.076818 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c877e242-dd4a-4905-9a14-068e18169b5f None admin] PATH_INFO: `/v3/services` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:18:14.076818 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c877e242-dd4a-4905-9a14-068e18169b5f None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:18:14.076818 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c877e242-dd4a-4905-9a14-068e18169b5f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:18:14.076818 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c877e242-dd4a-4905-9a14-068e18169b5f None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:18:14.085990 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 458/898] 10.222.0.30 () {56 vars in 1114 bytes} [Tue Sep 6 04:18:13 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:18:14.087020 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c877e242-dd4a-4905-9a14-068e18169b5f None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:18:14.117251 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b78b433-e4d6-4315-a54d-2ff2d5d6932b None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:18:14.160273 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f804a60-ac90-4370-ac82-a4f434efcfa2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:18:14.161049 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f804a60-ac90-4370-ac82-a4f434efcfa2 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:18:14.161417 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f804a60-ac90-4370-ac82-a4f434efcfa2 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:18:14.162488 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f804a60-ac90-4370-ac82-a4f434efcfa2 None admin] PATH_INFO: `/v3/projects` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:18:14.163202 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f804a60-ac90-4370-ac82-a4f434efcfa2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:18:14.163541 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f804a60-ac90-4370-ac82-a4f434efcfa2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:18:14.164973 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f804a60-ac90-4370-ac82-a4f434efcfa2 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:18:14.179033 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 441/899] 10.222.0.30 () {56 vars in 1148 bytes} [Tue Sep 6 04:18:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 04:18:14.180280 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0164a859-e61e-463b-8ba7-1247b2c19589 None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:18:14.180564 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0164a859-e61e-463b-8ba7-1247b2c19589 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:18:14.180839 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0164a859-e61e-463b-8ba7-1247b2c19589 None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:18:14.185349 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 442/900] 10.222.0.30 () {54 vars in 927 bytes} [Tue Sep 6 04:18:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 04:18:14.185349 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0164a859-e61e-463b-8ba7-1247b2c19589 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:18:14.200805 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c877e242-dd4a-4905-9a14-068e18169b5f None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:18:14.213337 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:18:14.238030 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee7ca93-385f-4b26-bc48-daa751aa2bea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:18:14.238647 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee7ca93-385f-4b26-bc48-daa751aa2bea None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:18:14.238931 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee7ca93-385f-4b26-bc48-daa751aa2bea None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:18:14.239208 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee7ca93-385f-4b26-bc48-daa751aa2bea None admin] PATH_INFO: `/v3/endpoints` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:18:14.239982 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee7ca93-385f-4b26-bc48-daa751aa2bea None admin] RBAC: Adding query filter params (interface=public, service_id=910e481bdab245edbf02abc160378d07, region_id=RegionOne) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:18:14.240291 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee7ca93-385f-4b26-bc48-daa751aa2bea None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:18:14.241694 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee7ca93-385f-4b26-bc48-daa751aa2bea None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:18:14.247871 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 459/901] 10.222.0.30 () {56 vars in 1251 bytes} [Tue Sep 6 04:18:14 2022] GET /identity/v3/endpoints?service_id=910e481bdab245edbf02abc160378d07&interface=public®ion_id=RegionOne => generated 194 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 04:18:14.248885 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9496cf71-05ab-4e49-ab0a-0d890658f287 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:18:14.249172 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9496cf71-05ab-4e49-ab0a-0d890658f287 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:18:14.249481 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9496cf71-05ab-4e49-ab0a-0d890658f287 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:18:14.268672 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-9496cf71-05ab-4e49-ab0a-0d890658f287 None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:18:14.269907 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-9496cf71-05ab-4e49-ab0a-0d890658f287 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:18:14.298215 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0164a859-e61e-463b-8ba7-1247b2c19589 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:18:14.323924 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:18:14.335900 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9cb6b0-7abb-44cf-9a02-7283188844a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:18:14.336512 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9cb6b0-7abb-44cf-9a02-7283188844a9 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:18:14.336796 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9cb6b0-7abb-44cf-9a02-7283188844a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:18:14.337070 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9cb6b0-7abb-44cf-9a02-7283188844a9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:18:14.337844 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9cb6b0-7abb-44cf-9a02-7283188844a9 None admin] RBAC: Adding query filter params (role.id=356631f6a0064bc695b78e3759d55d6a, scope.project.id=b707733ce65c42edbc1fd95ad0a1c06c, user.id=da5fed17101a4935bcc38b72caf9d263) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:18:14.338148 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9cb6b0-7abb-44cf-9a02-7283188844a9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:18:14.339577 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9cb6b0-7abb-44cf-9a02-7283188844a9 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:18:14.342911 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 460/902] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Sep 6 04:18:14 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 04:18:14.345753 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 443/903] 10.222.0.30 () {56 vars in 1374 bytes} [Tue Sep 6 04:18:14 2022] GET /identity/v3/role_assignments?user.id=da5fed17101a4935bcc38b72caf9d263&scope.project.id=b707733ce65c42edbc1fd95ad0a1c06c&role.id=356631f6a0064bc695b78e3759d55d6a => generated 259 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 04:18:14.351380 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9cb6b0-7abb-44cf-9a02-7283188844a9 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:18:14.416291 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177d8112-654e-4711-802f-93e7950c2d90 None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:18:14.416647 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177d8112-654e-4711-802f-93e7950c2d90 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:18:14.416843 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177d8112-654e-4711-802f-93e7950c2d90 None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:18:14.421242 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 461/904] 10.222.0.30 () {54 vars in 927 bytes} [Tue Sep 6 04:18:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 06 04:18:14.426913 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6eceef0-c281-41db-b5af-17bdeac523c1 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:18:14.427198 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6eceef0-c281-41db-b5af-17bdeac523c1 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:18:14.427537 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6eceef0-c281-41db-b5af-17bdeac523c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:18:14.449482 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-f6eceef0-c281-41db-b5af-17bdeac523c1 None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:18:14.451059 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-f6eceef0-c281-41db-b5af-17bdeac523c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:18:14.496180 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9cb6b0-7abb-44cf-9a02-7283188844a9 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:18:14.534585 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d645f7c-a49a-4d23-8516-1a2e12e850e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:18:14.535275 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d645f7c-a49a-4d23-8516-1a2e12e850e9 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:18:14.535550 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d645f7c-a49a-4d23-8516-1a2e12e850e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:18:14.535874 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d645f7c-a49a-4d23-8516-1a2e12e850e9 None admin] PATH_INFO: `/v3/services` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:18:14.536488 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d645f7c-a49a-4d23-8516-1a2e12e850e9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:18:14.537884 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d645f7c-a49a-4d23-8516-1a2e12e850e9 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:18:14.541374 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 462/905] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Sep 6 04:18:14 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 06 04:18:14.546726 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 444/906] 10.222.0.30 () {56 vars in 1087 bytes} [Tue Sep 6 04:18:14 2022] GET /identity/v3/services => generated 1268 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 04:18:14.624958 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6eceef0-c281-41db-b5af-17bdeac523c1 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:18:14.664135 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:18:14.735626 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6eceef0-c281-41db-b5af-17bdeac523c1 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:18:14.772080 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437e8865-e9be-478f-b9ab-d35521d8ab8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:18:14.772635 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437e8865-e9be-478f-b9ab-d35521d8ab8d None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:18:14.772914 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437e8865-e9be-478f-b9ab-d35521d8ab8d None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:18:14.773186 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437e8865-e9be-478f-b9ab-d35521d8ab8d None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:18:14.773807 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437e8865-e9be-478f-b9ab-d35521d8ab8d None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:18:14.775182 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437e8865-e9be-478f-b9ab-d35521d8ab8d None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:18:14.784765 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-437e8865-e9be-478f-b9ab-d35521d8ab8d None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 06 04:18:14.784765 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 04:18:14.784765 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 06 04:18:14.784765 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 06 04:18:14.784765 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 06 04:18:14.784765 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 06 04:18:14.784765 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 06 04:18:14.784765 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 04:18:14.784765 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 06 04:18:14.784765 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 04:18:14.784765 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 06 04:18:14.784765 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 04:18:14.784765 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 04:18:14.784765 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 04:18:14.784765 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 04:18:14.784765 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 04:18:14.785777 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 04:18:14.785777 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 04:18:14.785777 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 04:18:14.785777 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 04:18:14.785777 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 04:18:14.785777 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 04:18:14.785777 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 04:18:14.785777 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 06 04:18:14.785777 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 06 04:18:14.785777 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 06 04:18:14.785777 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 06 04:18:14.785777 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 04:18:14.785777 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 04:18:14.785777 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 04:18:14.785777 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 04:18:14.785777 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 04:18:14.785777 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 04:18:14.787077 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 04:18:14.787077 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 04:18:14.787077 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 04:18:14.787077 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 04:18:14.787077 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 04:18:14.787077 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 04:18:14.787077 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 04:18:14.787077 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 04:18:14.787077 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 04:18:14.787077 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 04:18:14.787077 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 06 04:18:14.787077 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 04:18:14.787077 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 06 04:18:14.787077 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 04:18:14.787077 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 463/907] 10.222.0.30 () {56 vars in 1132 bytes} [Tue Sep 6 04:18:14 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 04:18:14.794028 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d645f7c-a49a-4d23-8516-1a2e12e850e9 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:18:14.825661 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437e8865-e9be-478f-b9ab-d35521d8ab8d None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:18:14.849479 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:18:14.907041 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d645f7c-a49a-4d23-8516-1a2e12e850e9 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:18:14.944821 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8902f8cf-a72f-497e-91d5-5bcb25b11d8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:18:14.945938 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8902f8cf-a72f-497e-91d5-5bcb25b11d8e None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:18:14.946569 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8902f8cf-a72f-497e-91d5-5bcb25b11d8e None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:18:14.947038 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8902f8cf-a72f-497e-91d5-5bcb25b11d8e None admin] PATH_INFO: `/v3/services` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:18:14.947867 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8902f8cf-a72f-497e-91d5-5bcb25b11d8e None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:18:14.948379 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8902f8cf-a72f-497e-91d5-5bcb25b11d8e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:18:14.950112 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8902f8cf-a72f-497e-91d5-5bcb25b11d8e None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:18:14.951998 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:18:14.956669 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437e8865-e9be-478f-b9ab-d35521d8ab8d None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:18:14.957623 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 445/908] 10.222.0.30 () {56 vars in 1126 bytes} [Tue Sep 6 04:18:14 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:18:14.965900 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8902f8cf-a72f-497e-91d5-5bcb25b11d8e None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:18:15.001179 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b3f3c4-fa83-489a-8b62-6f3d8391acba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:18:15.002401 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b3f3c4-fa83-489a-8b62-6f3d8391acba None admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:18:15.002824 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b3f3c4-fa83-489a-8b62-6f3d8391acba None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:18:15.003239 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b3f3c4-fa83-489a-8b62-6f3d8391acba None admin] PATH_INFO: `/v3/services` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:18:15.004043 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b3f3c4-fa83-489a-8b62-6f3d8391acba None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:18:15.006014 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b3f3c4-fa83-489a-8b62-6f3d8391acba None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:18:15.069408 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8902f8cf-a72f-497e-91d5-5bcb25b11d8e None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:18:15.084181 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:18:15.101491 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 464/909] 10.222.0.30 () {60 vars in 1141 bytes} [Tue Sep 6 04:18:14 2022] POST /identity/v3/services => generated 241 bytes in 277 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 04:18:15.129003 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c7af64-d4bb-40c6-a295-5e9005b905ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:18:15.132836 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c7af64-d4bb-40c6-a295-5e9005b905ed None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:18:15.134725 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c7af64-d4bb-40c6-a295-5e9005b905ed None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:18:15.135296 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c7af64-d4bb-40c6-a295-5e9005b905ed None admin] PATH_INFO: `/v3/services` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:18:15.136190 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c7af64-d4bb-40c6-a295-5e9005b905ed None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:18:15.136703 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c7af64-d4bb-40c6-a295-5e9005b905ed None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:18:15.138369 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c7af64-d4bb-40c6-a295-5e9005b905ed None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:18:15.145930 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 446/910] 10.222.0.30 () {56 vars in 1126 bytes} [Tue Sep 6 04:18:14 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:18:15.151655 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b3f3c4-fa83-489a-8b62-6f3d8391acba None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:18:15.270086 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b3f3c4-fa83-489a-8b62-6f3d8391acba None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:18:15.305433 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418a1f0d-5463-4b02-8a29-489f6f4767ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:18:15.306369 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418a1f0d-5463-4b02-8a29-489f6f4767ad None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:18:15.306851 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418a1f0d-5463-4b02-8a29-489f6f4767ad None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:18:15.307338 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418a1f0d-5463-4b02-8a29-489f6f4767ad None admin] PATH_INFO: `/v3/endpoints` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:18:15.308046 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418a1f0d-5463-4b02-8a29-489f6f4767ad None admin] RBAC: Adding query filter params (interface=public, service_id=74baf84da33142f7a5ab1bdc07f907a7, region_id=RegionOne) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:18:15.308353 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418a1f0d-5463-4b02-8a29-489f6f4767ad None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:18:15.309758 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418a1f0d-5463-4b02-8a29-489f6f4767ad None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:18:15.316579 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 465/911] 10.222.0.30 () {56 vars in 1251 bytes} [Tue Sep 6 04:18:15 2022] GET /identity/v3/endpoints?service_id=74baf84da33142f7a5ab1bdc07f907a7&interface=public®ion_id=RegionOne => generated 194 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 04:18:15.322039 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c7af64-d4bb-40c6-a295-5e9005b905ed None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:18:15.425715 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c7af64-d4bb-40c6-a295-5e9005b905ed None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:18:15.460620 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6489e3d7-b8b4-4bff-91fd-c8c47ce91b54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:18:15.461266 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6489e3d7-b8b4-4bff-91fd-c8c47ce91b54 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:18:15.461582 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6489e3d7-b8b4-4bff-91fd-c8c47ce91b54 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:18:15.461934 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6489e3d7-b8b4-4bff-91fd-c8c47ce91b54 None admin] PATH_INFO: `/v3/services` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:18:15.462571 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6489e3d7-b8b4-4bff-91fd-c8c47ce91b54 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:18:15.464056 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6489e3d7-b8b4-4bff-91fd-c8c47ce91b54 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:18:15.471871 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 447/912] 10.222.0.30 () {56 vars in 1087 bytes} [Tue Sep 6 04:18:15 2022] GET /identity/v3/services => generated 1497 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 04:18:16.318130 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d98453f-6ad0-466c-9575-35c020309553 None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:18:16.318647 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d98453f-6ad0-466c-9575-35c020309553 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:18:16.318724 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d98453f-6ad0-466c-9575-35c020309553 None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:18:16.320272 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 466/913] 10.222.0.30 () {54 vars in 927 bytes} [Tue Sep 6 04:18:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 04:18:16.325566 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47b1f9e-0c6e-4b9e-ad79-79a9cf5c6431 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:18:16.325977 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47b1f9e-0c6e-4b9e-ad79-79a9cf5c6431 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:18:16.326148 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47b1f9e-0c6e-4b9e-ad79-79a9cf5c6431 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:18:16.344851 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-c47b1f9e-0c6e-4b9e-ad79-79a9cf5c6431 None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:18:16.346162 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-c47b1f9e-0c6e-4b9e-ad79-79a9cf5c6431 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:18:16.381734 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:18:16.420002 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 448/914] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Sep 6 04:18:16 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 04:18:16.567730 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad830f4-c392-44e0-8497-ec4515d60867 None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:18:16.568085 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad830f4-c392-44e0-8497-ec4515d60867 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:18:16.568331 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad830f4-c392-44e0-8497-ec4515d60867 None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:18:16.569980 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 467/915] 10.222.0.30 () {54 vars in 927 bytes} [Tue Sep 6 04:18:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 04:18:16.575673 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e612232-8ccc-40d8-bacb-00d82a9841f8 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:18:16.575993 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e612232-8ccc-40d8-bacb-00d82a9841f8 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:18:16.576232 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e612232-8ccc-40d8-bacb-00d82a9841f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:18:16.595644 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-0e612232-8ccc-40d8-bacb-00d82a9841f8 None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:18:16.597056 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-0e612232-8ccc-40d8-bacb-00d82a9841f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:18:16.671530 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 449/916] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Sep 6 04:18:16 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 04:18:16.693258 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad830f4-c392-44e0-8497-ec4515d60867 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:18:16.719165 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:18:16.803323 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad830f4-c392-44e0-8497-ec4515d60867 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:18:16.840732 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238251c2-bbd4-4ed8-a765-dbda11a68e89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:18:16.841290 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238251c2-bbd4-4ed8-a765-dbda11a68e89 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:18:16.841636 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238251c2-bbd4-4ed8-a765-dbda11a68e89 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:18:16.841905 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238251c2-bbd4-4ed8-a765-dbda11a68e89 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:18:16.852264 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238251c2-bbd4-4ed8-a765-dbda11a68e89 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 04:18:16.857279 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-238251c2-bbd4-4ed8-a765-dbda11a68e89 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 06 04:18:16.857279 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 04:18:16.857279 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 04:18:16.857279 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 04:18:16.857279 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 04:18:16.857279 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 04:18:16.857279 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 04:18:16.857279 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 04:18:16.857279 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 04:18:16.857279 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 04:18:16.857279 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 04:18:16.857279 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 04:18:16.857279 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 04:18:16.857279 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 04:18:16.857279 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 04:18:16.857279 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 04:18:16.858273 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 04:18:16.858273 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 04:18:16.858273 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 04:18:16.858273 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 04:18:16.858273 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 04:18:16.858273 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 04:18:16.858273 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 04:18:16.858273 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 04:18:16.858273 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 04:18:16.858273 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 04:18:16.858273 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 04:18:16.858273 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 04:18:16.858273 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 04:18:16.858273 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 04:18:16.858273 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 04:18:16.858273 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 04:18:16.859128 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 04:18:16.859128 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 04:18:16.859128 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 04:18:16.859128 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 04:18:16.859128 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 04:18:16.859128 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 04:18:16.859128 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 04:18:16.859128 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 04:18:16.859128 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 04:18:16.859128 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 04:18:16.859128 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 06 04:18:16.859128 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 04:18:16.859128 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 468/917] 10.222.0.30 () {56 vars in 1120 bytes} [Tue Sep 6 04:18:16 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 04:18:16.864655 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e612232-8ccc-40d8-bacb-00d82a9841f8 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:18:16.954278 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238251c2-bbd4-4ed8-a765-dbda11a68e89 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:18:16.973860 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:18:16.976479 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e612232-8ccc-40d8-bacb-00d82a9841f8 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:18:17.023259 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e0ae8d-5444-43a5-a1ef-e45955f7b529 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:18:17.023879 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e0ae8d-5444-43a5-a1ef-e45955f7b529 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:18:17.024158 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e0ae8d-5444-43a5-a1ef-e45955f7b529 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:18:17.024431 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e0ae8d-5444-43a5-a1ef-e45955f7b529 None admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:18:17.025101 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e0ae8d-5444-43a5-a1ef-e45955f7b529 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:18:17.025434 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e0ae8d-5444-43a5-a1ef-e45955f7b529 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:18:17.027677 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e0ae8d-5444-43a5-a1ef-e45955f7b529 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:18:17.043142 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 450/918] 10.222.0.30 () {56 vars in 1115 bytes} [Tue Sep 6 04:18:16 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:18:17.044774 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e0ae8d-5444-43a5-a1ef-e45955f7b529 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:18:17.046488 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:18:17.067092 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238251c2-bbd4-4ed8-a765-dbda11a68e89 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:18:17.105691 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18da1bc9-1ab9-4a0a-b7d6-cebd7de6147b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:18:17.106674 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18da1bc9-1ab9-4a0a-b7d6-cebd7de6147b None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:18:17.107161 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18da1bc9-1ab9-4a0a-b7d6-cebd7de6147b None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:18:17.107635 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18da1bc9-1ab9-4a0a-b7d6-cebd7de6147b None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:18:17.108440 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18da1bc9-1ab9-4a0a-b7d6-cebd7de6147b None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:18:17.110040 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18da1bc9-1ab9-4a0a-b7d6-cebd7de6147b None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:18:17.120306 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-18da1bc9-1ab9-4a0a-b7d6-cebd7de6147b None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 06 04:18:17.120306 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 04:18:17.120306 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 06 04:18:17.120306 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 06 04:18:17.120306 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 06 04:18:17.120306 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 06 04:18:17.120306 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 06 04:18:17.120306 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 04:18:17.120306 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 06 04:18:17.120306 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 04:18:17.120306 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 06 04:18:17.120306 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 04:18:17.120306 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 04:18:17.120306 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 04:18:17.120306 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 04:18:17.120306 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 04:18:17.120306 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 04:18:17.121397 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 04:18:17.121397 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 04:18:17.121397 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 04:18:17.121397 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 04:18:17.121397 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 04:18:17.121397 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 04:18:17.121397 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 06 04:18:17.121397 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 06 04:18:17.121397 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 06 04:18:17.121397 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 06 04:18:17.121397 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 04:18:17.121397 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 04:18:17.121397 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 04:18:17.121397 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 04:18:17.121397 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 04:18:17.121397 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 04:18:17.122245 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 04:18:17.122245 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 04:18:17.122245 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 04:18:17.122245 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 04:18:17.122245 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 04:18:17.122245 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 04:18:17.122245 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 04:18:17.122245 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 04:18:17.122245 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 04:18:17.122245 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 04:18:17.122245 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 06 04:18:17.122245 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 04:18:17.122245 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 06 04:18:17.122245 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 04:18:17.130106 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 469/919] 10.222.0.30 () {56 vars in 1114 bytes} [Tue Sep 6 04:18:16 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 04:18:17.131255 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18da1bc9-1ab9-4a0a-b7d6-cebd7de6147b None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:18:17.172483 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e0ae8d-5444-43a5-a1ef-e45955f7b529 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:18:17.209857 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa2ed26-5e8c-48c3-8353-0b160043223d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:18:17.210492 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa2ed26-5e8c-48c3-8353-0b160043223d None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:18:17.210766 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa2ed26-5e8c-48c3-8353-0b160043223d None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:18:17.211033 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa2ed26-5e8c-48c3-8353-0b160043223d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:18:17.224590 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa2ed26-5e8c-48c3-8353-0b160043223d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:18:17.226408 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa2ed26-5e8c-48c3-8353-0b160043223d None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:18:17.241131 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-4aa2ed26-5e8c-48c3-8353-0b160043223d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 04:18:17.241131 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 04:18:17.241131 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 04:18:17.241131 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 04:18:17.241131 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 04:18:17.241131 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 04:18:17.241131 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 04:18:17.241131 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 04:18:17.241131 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 04:18:17.241131 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 04:18:17.241131 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 04:18:17.241131 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 04:18:17.241131 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 04:18:17.241131 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 04:18:17.241131 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 04:18:17.241131 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 04:18:17.242138 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 04:18:17.242138 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 04:18:17.242138 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 04:18:17.242138 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 04:18:17.242138 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 04:18:17.242138 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 04:18:17.242138 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 04:18:17.242138 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 04:18:17.242138 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 04:18:17.242138 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 04:18:17.242138 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 04:18:17.242138 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 04:18:17.242138 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 04:18:17.242138 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 04:18:17.242138 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 04:18:17.242138 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 04:18:17.242138 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 04:18:17.243040 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 04:18:17.243040 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 04:18:17.243040 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 04:18:17.243040 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 451/920] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Sep 6 04:18:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 04:18:17.243690 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578ecc56-500f-4896-9142-4c4908edc8f0 None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:18:17.243690 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578ecc56-500f-4896-9142-4c4908edc8f0 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:18:17.243857 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578ecc56-500f-4896-9142-4c4908edc8f0 None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:18:17.245196 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 452/921] 10.222.0.30 () {54 vars in 927 bytes} [Tue Sep 6 04:18:17 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 04:18:17.247909 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578ecc56-500f-4896-9142-4c4908edc8f0 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:18:17.253427 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18da1bc9-1ab9-4a0a-b7d6-cebd7de6147b None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:18:17.265772 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:18:17.289813 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7943938-7aec-4719-a6f0-726a2d4fd5f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:18:17.290425 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7943938-7aec-4719-a6f0-726a2d4fd5f8 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:18:17.290706 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7943938-7aec-4719-a6f0-726a2d4fd5f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:18:17.290967 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7943938-7aec-4719-a6f0-726a2d4fd5f8 None admin] PATH_INFO: `/v3/services` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:18:17.291594 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7943938-7aec-4719-a6f0-726a2d4fd5f8 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:18:17.291891 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7943938-7aec-4719-a6f0-726a2d4fd5f8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:18:17.293335 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7943938-7aec-4719-a6f0-726a2d4fd5f8 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:18:17.301429 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 470/922] 10.222.0.30 () {56 vars in 1114 bytes} [Tue Sep 6 04:18:17 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:18:17.301429 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce7613e-bb4b-4a83-8e88-f5861ae566cc None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:18:17.301429 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce7613e-bb4b-4a83-8e88-f5861ae566cc None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:18:17.301730 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce7613e-bb4b-4a83-8e88-f5861ae566cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:18:17.321140 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-6ce7613e-bb4b-4a83-8e88-f5861ae566cc None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:18:17.322352 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-6ce7613e-bb4b-4a83-8e88-f5861ae566cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:18:17.326040 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:18:17.364084 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578ecc56-500f-4896-9142-4c4908edc8f0 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:18:17.398881 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 471/923] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Sep 6 04:18:17 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 04:18:17.399997 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce7613e-bb4b-4a83-8e88-f5861ae566cc None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:18:17.410170 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26106b77-975c-438e-83a7-dd4a39c3a6ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:18:17.411075 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26106b77-975c-438e-83a7-dd4a39c3a6ee None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:18:17.411550 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26106b77-975c-438e-83a7-dd4a39c3a6ee None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:18:17.412038 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26106b77-975c-438e-83a7-dd4a39c3a6ee None admin] PATH_INFO: `/v3/domains` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:18:17.412885 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26106b77-975c-438e-83a7-dd4a39c3a6ee None admin] RBAC: Adding query filter params (name=Default) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:18:17.413409 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26106b77-975c-438e-83a7-dd4a39c3a6ee None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:18:17.415115 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26106b77-975c-438e-83a7-dd4a39c3a6ee None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:18:17.428815 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 453/924] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Sep 6 04:18:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 04:18:17.436136 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26106b77-975c-438e-83a7-dd4a39c3a6ee None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:18:17.512053 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:18:17.530247 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce7613e-bb4b-4a83-8e88-f5861ae566cc None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:18:17.541625 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26106b77-975c-438e-83a7-dd4a39c3a6ee None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:18:17.567593 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8123b76c-bb75-40d6-84fb-f50132130f0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:18:17.568200 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8123b76c-bb75-40d6-84fb-f50132130f0e None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:18:17.568476 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8123b76c-bb75-40d6-84fb-f50132130f0e None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:18:17.568746 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8123b76c-bb75-40d6-84fb-f50132130f0e None admin] PATH_INFO: `/v3/services` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:18:17.569400 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8123b76c-bb75-40d6-84fb-f50132130f0e None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:18:17.569700 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8123b76c-bb75-40d6-84fb-f50132130f0e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:18:17.571095 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8123b76c-bb75-40d6-84fb-f50132130f0e None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:18:17.580524 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 472/925] 10.222.0.30 () {56 vars in 1114 bytes} [Tue Sep 6 04:18:17 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:18:17.580524 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edaf614-7f9f-4a47-876b-12ce7c7b5a93 None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:18:17.580524 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edaf614-7f9f-4a47-876b-12ce7c7b5a93 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:18:17.580524 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edaf614-7f9f-4a47-876b-12ce7c7b5a93 None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:18:17.580524 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 473/926] 10.222.0.30 () {54 vars in 927 bytes} [Tue Sep 6 04:18:17 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 04:18:17.582018 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7edaf614-7f9f-4a47-876b-12ce7c7b5a93 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:18:17.584726 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18839b5-033e-492b-a5a3-a01c83a7bc62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:18:17.586172 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18839b5-033e-492b-a5a3-a01c83a7bc62 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:18:17.586172 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18839b5-033e-492b-a5a3-a01c83a7bc62 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:18:17.586172 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18839b5-033e-492b-a5a3-a01c83a7bc62 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:18:17.586972 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:18:17.595352 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18839b5-033e-492b-a5a3-a01c83a7bc62 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:18:17.597683 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18839b5-033e-492b-a5a3-a01c83a7bc62 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:18:17.609872 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-b18839b5-033e-492b-a5a3-a01c83a7bc62 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 06 04:18:17.609872 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 04:18:17.609872 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 04:18:17.609872 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 04:18:17.609872 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 04:18:17.609872 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 04:18:17.609872 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 04:18:17.609872 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 04:18:17.609872 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 04:18:17.609872 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 04:18:17.609872 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 04:18:17.609872 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 04:18:17.609872 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 06 04:18:17.609872 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 06 04:18:17.609872 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 06 04:18:17.609872 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 06 04:18:17.610808 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 04:18:17.610808 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 04:18:17.610808 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 06 04:18:17.610808 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 04:18:17.610808 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 06 04:18:17.610808 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 04:18:17.610808 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 04:18:17.610808 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 04:18:17.610808 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 04:18:17.610808 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 04:18:17.610808 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 04:18:17.610808 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 04:18:17.610808 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 04:18:17.610808 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 04:18:17.610808 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 04:18:17.610808 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 04:18:17.610808 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 04:18:17.610808 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 04:18:17.611847 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 04:18:17.611847 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 04:18:17.611847 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 06 04:18:17.611847 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 06 04:18:17.611847 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 06 04:18:17.611847 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 06 04:18:17.611847 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 06 04:18:17.611847 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 06 04:18:17.611847 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 06 04:18:17.611847 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 04:18:17.616158 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 454/927] 10.222.0.30 () {56 vars in 1117 bytes} [Tue Sep 6 04:18:17 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 04:18:17.617376 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8539188f-981c-4514-81fc-881ecc922eeb None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:18:17.617662 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8539188f-981c-4514-81fc-881ecc922eeb None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:18:17.617941 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8539188f-981c-4514-81fc-881ecc922eeb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:18:17.637899 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-8539188f-981c-4514-81fc-881ecc922eeb None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:18:17.639101 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-8539188f-981c-4514-81fc-881ecc922eeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:18:17.693253 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7edaf614-7f9f-4a47-876b-12ce7c7b5a93 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:18:17.706311 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 455/928] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Sep 6 04:18:17 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 04:18:17.707157 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8539188f-981c-4514-81fc-881ecc922eeb None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:18:17.737425 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131f9a0a-1251-434b-94eb-417a63e671d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:18:17.738383 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131f9a0a-1251-434b-94eb-417a63e671d7 None admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:18:17.738668 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131f9a0a-1251-434b-94eb-417a63e671d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:18:17.738945 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131f9a0a-1251-434b-94eb-417a63e671d7 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:18:17.739612 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131f9a0a-1251-434b-94eb-417a63e671d7 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:18:17.740982 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131f9a0a-1251-434b-94eb-417a63e671d7 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:18:17.769978 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:18:17.807041 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8539188f-981c-4514-81fc-881ecc922eeb None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:18:17.821335 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 474/929] 10.222.0.30 () {60 vars in 1144 bytes} [Tue Sep 6 04:18:17 2022] POST /identity/v3/endpoints => generated 346 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 04:18:17.821335 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131f9a0a-1251-434b-94eb-417a63e671d7 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:18:17.853648 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d49a30-4cf0-4a78-837f-0423ec3ea058 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:18:17.854275 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d49a30-4cf0-4a78-837f-0423ec3ea058 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:18:17.854554 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d49a30-4cf0-4a78-837f-0423ec3ea058 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:18:17.854825 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d49a30-4cf0-4a78-837f-0423ec3ea058 None admin] PATH_INFO: `/v3/users` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:18:17.855560 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d49a30-4cf0-4a78-837f-0423ec3ea058 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:18:17.855868 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d49a30-4cf0-4a78-837f-0423ec3ea058 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:18:17.857411 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d49a30-4cf0-4a78-837f-0423ec3ea058 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:18:17.883337 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 456/930] 10.222.0.30 () {56 vars in 1149 bytes} [Tue Sep 6 04:18:17 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:18:17.889272 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d49a30-4cf0-4a78-837f-0423ec3ea058 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:18:17.915231 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:18:17.966133 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131f9a0a-1251-434b-94eb-417a63e671d7 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:18:18.010968 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f2305d-0c24-4f44-a195-3b8579a18495 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:18:18.011774 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f2305d-0c24-4f44-a195-3b8579a18495 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:18:18.012053 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f2305d-0c24-4f44-a195-3b8579a18495 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:18:18.012662 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f2305d-0c24-4f44-a195-3b8579a18495 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:18:18.013607 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f2305d-0c24-4f44-a195-3b8579a18495 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:18:18.015448 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f2305d-0c24-4f44-a195-3b8579a18495 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:18:18.023470 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d49a30-4cf0-4a78-837f-0423ec3ea058 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:18:18.026875 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-96f2305d-0c24-4f44-a195-3b8579a18495 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 06 04:18:18.026875 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 04:18:18.026875 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 06 04:18:18.026875 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 06 04:18:18.026875 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 06 04:18:18.026875 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 06 04:18:18.026875 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 06 04:18:18.026875 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 04:18:18.026875 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 06 04:18:18.026875 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 04:18:18.026875 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 06 04:18:18.026875 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 04:18:18.026875 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 04:18:18.026875 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 04:18:18.026875 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 04:18:18.026875 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 04:18:18.026875 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 04:18:18.028116 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 04:18:18.028116 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 04:18:18.028116 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 04:18:18.028116 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 04:18:18.028116 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 04:18:18.028116 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 04:18:18.028116 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 06 04:18:18.028116 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 06 04:18:18.028116 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 06 04:18:18.028116 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 06 04:18:18.028116 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 04:18:18.028116 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 04:18:18.028116 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 04:18:18.028116 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 04:18:18.028116 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 04:18:18.028116 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 04:18:18.029508 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 04:18:18.029508 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 04:18:18.029508 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 04:18:18.029508 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 04:18:18.029508 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 04:18:18.029508 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 04:18:18.029508 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 04:18:18.029508 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 04:18:18.029508 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 04:18:18.029508 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 04:18:18.029508 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 06 04:18:18.029508 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 04:18:18.029508 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 06 04:18:18.029508 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 04:18:18.029508 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 475/931] 10.222.0.30 () {56 vars in 1126 bytes} [Tue Sep 6 04:18:17 2022] GET /identity/v3/services/object-store => generated 93 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 04:18:18.030389 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f2305d-0c24-4f44-a195-3b8579a18495 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:18:18.046379 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:18:18.065238 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15233d66-a5c5-4213-8486-c1fa47156c1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:18:18.066096 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15233d66-a5c5-4213-8486-c1fa47156c1f None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:18:18.066445 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15233d66-a5c5-4213-8486-c1fa47156c1f None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:18:18.066783 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15233d66-a5c5-4213-8486-c1fa47156c1f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:18:18.077328 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15233d66-a5c5-4213-8486-c1fa47156c1f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:18:18.078762 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15233d66-a5c5-4213-8486-c1fa47156c1f None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:18:18.093295 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-15233d66-a5c5-4213-8486-c1fa47156c1f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 04:18:18.093295 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 04:18:18.093295 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 04:18:18.093295 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 04:18:18.093295 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 04:18:18.093295 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 04:18:18.093295 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 04:18:18.093295 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 04:18:18.093295 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 04:18:18.093295 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 04:18:18.093295 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 04:18:18.093295 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 04:18:18.093295 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 04:18:18.093295 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 04:18:18.093295 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 04:18:18.093295 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 04:18:18.094309 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 04:18:18.094309 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 04:18:18.094309 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 04:18:18.094309 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 04:18:18.094309 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 04:18:18.094309 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 04:18:18.094309 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 04:18:18.094309 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 04:18:18.094309 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 04:18:18.094309 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 04:18:18.094309 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 04:18:18.094309 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 04:18:18.094309 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 04:18:18.094309 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 04:18:18.094309 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 04:18:18.094309 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 04:18:18.094309 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 04:18:18.095214 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 04:18:18.095214 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 04:18:18.095214 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 04:18:18.095214 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 457/932] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Sep 6 04:18:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 04:18:18.095214 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15233d66-a5c5-4213-8486-c1fa47156c1f None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:18:18.141343 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f2305d-0c24-4f44-a195-3b8579a18495 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:18:18.165619 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:18:18.177196 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986f1db0-340f-43e7-a679-296be3bb14cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:18:18.181360 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986f1db0-340f-43e7-a679-296be3bb14cf None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:18:18.181360 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986f1db0-340f-43e7-a679-296be3bb14cf None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:18:18.181360 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986f1db0-340f-43e7-a679-296be3bb14cf None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:18:18.181360 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-986f1db0-340f-43e7-a679-296be3bb14cf None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:18:18.181360 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-986f1db0-340f-43e7-a679-296be3bb14cf None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:18:18.189066 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-986f1db0-340f-43e7-a679-296be3bb14cf None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 06 04:18:18.189066 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 04:18:18.189066 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 06 04:18:18.189066 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 06 04:18:18.189066 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 06 04:18:18.189066 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 06 04:18:18.189066 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 06 04:18:18.189066 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 04:18:18.189066 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 06 04:18:18.189066 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 04:18:18.189066 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 06 04:18:18.189066 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 04:18:18.189066 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 04:18:18.189066 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 04:18:18.189066 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 04:18:18.189066 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 04:18:18.190278 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 04:18:18.190278 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 04:18:18.190278 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 04:18:18.190278 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 04:18:18.190278 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 04:18:18.190278 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 04:18:18.190278 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 04:18:18.190278 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 06 04:18:18.190278 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 06 04:18:18.190278 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 06 04:18:18.190278 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 06 04:18:18.190278 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 04:18:18.190278 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 04:18:18.190278 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 04:18:18.190278 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 04:18:18.190278 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 04:18:18.190278 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 04:18:18.191253 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 04:18:18.191253 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 04:18:18.191253 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 04:18:18.191253 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 04:18:18.191253 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 04:18:18.191253 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 04:18:18.191253 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 04:18:18.191253 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 04:18:18.191253 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 04:18:18.191253 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 04:18:18.191253 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 06 04:18:18.191253 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 04:18:18.191253 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 06 04:18:18.191253 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 04:18:18.191253 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 476/933] 10.222.0.30 () {56 vars in 1132 bytes} [Tue Sep 6 04:18:18 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 04:18:18.192040 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986f1db0-340f-43e7-a679-296be3bb14cf None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:18:18.212473 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15233d66-a5c5-4213-8486-c1fa47156c1f None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:18:18.247533 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0943179a-2191-4439-8606-55fe8f271b92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:18:18.248386 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0943179a-2191-4439-8606-55fe8f271b92 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:18:18.248816 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0943179a-2191-4439-8606-55fe8f271b92 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:18:18.249692 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0943179a-2191-4439-8606-55fe8f271b92 None admin] PATH_INFO: `/v3/services` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:18:18.250756 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0943179a-2191-4439-8606-55fe8f271b92 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:18:18.251234 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0943179a-2191-4439-8606-55fe8f271b92 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:18:18.254180 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0943179a-2191-4439-8606-55fe8f271b92 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:18:18.260601 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 458/934] 10.222.0.30 () {56 vars in 1122 bytes} [Tue Sep 6 04:18:18 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:18:18.261414 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0943179a-2191-4439-8606-55fe8f271b92 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:18:18.300192 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986f1db0-340f-43e7-a679-296be3bb14cf None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:18:18.301070 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:18:18.336890 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a52f601-8cdc-49fb-92e4-f0458d6e4283 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:18:18.337723 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a52f601-8cdc-49fb-92e4-f0458d6e4283 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:18:18.338002 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a52f601-8cdc-49fb-92e4-f0458d6e4283 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:18:18.338761 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a52f601-8cdc-49fb-92e4-f0458d6e4283 None admin] PATH_INFO: `/v3/domains` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:18:18.339420 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a52f601-8cdc-49fb-92e4-f0458d6e4283 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:18:18.339722 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a52f601-8cdc-49fb-92e4-f0458d6e4283 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:18:18.341412 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a52f601-8cdc-49fb-92e4-f0458d6e4283 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:18:18.356913 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 477/935] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Sep 6 04:18:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:18:18.357782 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a52f601-8cdc-49fb-92e4-f0458d6e4283 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:18:18.378109 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0943179a-2191-4439-8606-55fe8f271b92 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:18:18.417198 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:18:18.419748 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a1d874-0838-4fc0-b8ae-1691b02dcbc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:18:18.423008 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a1d874-0838-4fc0-b8ae-1691b02dcbc0 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:18:18.423008 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a1d874-0838-4fc0-b8ae-1691b02dcbc0 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:18:18.423008 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a1d874-0838-4fc0-b8ae-1691b02dcbc0 None admin] PATH_INFO: `/v3/services` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:18:18.423008 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a1d874-0838-4fc0-b8ae-1691b02dcbc0 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:18:18.423008 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a1d874-0838-4fc0-b8ae-1691b02dcbc0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:18:18.425380 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a1d874-0838-4fc0-b8ae-1691b02dcbc0 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:18:18.430905 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 459/936] 10.222.0.30 () {56 vars in 1126 bytes} [Tue Sep 6 04:18:18 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:18:18.431696 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a1d874-0838-4fc0-b8ae-1691b02dcbc0 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:18:18.475119 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a52f601-8cdc-49fb-92e4-f0458d6e4283 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:18:18.516138 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed966371-0e08-4535-a701-9e89dfb7350b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:18:18.517864 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed966371-0e08-4535-a701-9e89dfb7350b None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:18:18.518367 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed966371-0e08-4535-a701-9e89dfb7350b None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:18:18.518880 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed966371-0e08-4535-a701-9e89dfb7350b None admin] PATH_INFO: `/v3/services` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:18:18.519689 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed966371-0e08-4535-a701-9e89dfb7350b None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:18:18.520169 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed966371-0e08-4535-a701-9e89dfb7350b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:18:18.521755 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed966371-0e08-4535-a701-9e89dfb7350b None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:18:18.529258 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 478/937] 10.222.0.30 () {56 vars in 1122 bytes} [Tue Sep 6 04:18:18 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:18:18.530579 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed966371-0e08-4535-a701-9e89dfb7350b None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:18:18.543621 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:18:18.550312 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a1d874-0838-4fc0-b8ae-1691b02dcbc0 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:18:18.586058 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1218ddbd-27a0-4816-8cdc-f677e08c9cb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:18:18.586691 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1218ddbd-27a0-4816-8cdc-f677e08c9cb5 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:18:18.586968 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1218ddbd-27a0-4816-8cdc-f677e08c9cb5 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:18:18.587236 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1218ddbd-27a0-4816-8cdc-f677e08c9cb5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:18:18.593966 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1218ddbd-27a0-4816-8cdc-f677e08c9cb5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:18:18.595383 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1218ddbd-27a0-4816-8cdc-f677e08c9cb5 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:18:18.606812 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-1218ddbd-27a0-4816-8cdc-f677e08c9cb5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 04:18:18.606812 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 04:18:18.606812 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 04:18:18.606812 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 04:18:18.606812 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 04:18:18.606812 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 04:18:18.606812 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 04:18:18.606812 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 04:18:18.606812 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 04:18:18.606812 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 04:18:18.606812 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 04:18:18.606812 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 04:18:18.606812 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 06 04:18:18.606812 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 06 04:18:18.606812 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 06 04:18:18.607753 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 06 04:18:18.607753 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 04:18:18.607753 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 04:18:18.607753 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 04:18:18.607753 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 04:18:18.607753 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 04:18:18.607753 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 04:18:18.607753 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 04:18:18.607753 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 04:18:18.607753 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 04:18:18.607753 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 04:18:18.607753 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 04:18:18.607753 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 04:18:18.607753 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 04:18:18.607753 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 04:18:18.607753 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 04:18:18.607753 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 04:18:18.608696 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 06 04:18:18.608696 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 06 04:18:18.608696 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 06 04:18:18.608696 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 06 04:18:18.608696 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 06 04:18:18.608696 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 06 04:18:18.608696 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 04:18:18.608696 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 04:18:18.608696 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 460/938] 10.222.0.30 () {56 vars in 1111 bytes} [Tue Sep 6 04:18:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 04:18:18.609181 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1218ddbd-27a0-4816-8cdc-f677e08c9cb5 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:18:18.640741 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed966371-0e08-4535-a701-9e89dfb7350b None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:18:18.655093 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:18:18.677075 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bea28f1-0e8d-4ac8-a343-988ea1ea2461 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:18:18.677645 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bea28f1-0e8d-4ac8-a343-988ea1ea2461 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:18:18.677925 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bea28f1-0e8d-4ac8-a343-988ea1ea2461 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:18:18.678189 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bea28f1-0e8d-4ac8-a343-988ea1ea2461 None admin] PATH_INFO: `/v3/services` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:18:18.678816 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bea28f1-0e8d-4ac8-a343-988ea1ea2461 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:18:18.679117 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bea28f1-0e8d-4ac8-a343-988ea1ea2461 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:18:18.680579 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bea28f1-0e8d-4ac8-a343-988ea1ea2461 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:18:18.685436 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 479/939] 10.222.0.30 () {56 vars in 1126 bytes} [Tue Sep 6 04:18:18 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:18:18.686467 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bea28f1-0e8d-4ac8-a343-988ea1ea2461 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:18:18.723625 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1218ddbd-27a0-4816-8cdc-f677e08c9cb5 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:18:18.760376 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95dc9ba4-091b-4d31-8a9e-0a0b6c62a4d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:18:18.761300 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95dc9ba4-091b-4d31-8a9e-0a0b6c62a4d4 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:18:18.761832 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95dc9ba4-091b-4d31-8a9e-0a0b6c62a4d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:18:18.762107 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95dc9ba4-091b-4d31-8a9e-0a0b6c62a4d4 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:18:18.762820 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95dc9ba4-091b-4d31-8a9e-0a0b6c62a4d4 None admin] RBAC: Adding query filter params (interface=public, service_id=ee9be98afa174081a4bd7524fb631214, region_id=RegionOne) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:18:18.763127 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95dc9ba4-091b-4d31-8a9e-0a0b6c62a4d4 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:18:18.764486 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95dc9ba4-091b-4d31-8a9e-0a0b6c62a4d4 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:18:18.774857 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 461/940] 10.222.0.30 () {56 vars in 1251 bytes} [Tue Sep 6 04:18:18 2022] GET /identity/v3/endpoints?service_id=ee9be98afa174081a4bd7524fb631214&interface=public®ion_id=RegionOne => generated 194 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:18:18.774857 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95dc9ba4-091b-4d31-8a9e-0a0b6c62a4d4 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:18:18.796406 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:18:18.797610 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bea28f1-0e8d-4ac8-a343-988ea1ea2461 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:18:18.832754 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087d40ff-fdce-4b02-8640-6e5250fe4871 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:18:18.833393 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087d40ff-fdce-4b02-8640-6e5250fe4871 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:18:18.833678 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087d40ff-fdce-4b02-8640-6e5250fe4871 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:18:18.833951 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087d40ff-fdce-4b02-8640-6e5250fe4871 None admin] PATH_INFO: `/v3/projects` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:18:18.834645 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087d40ff-fdce-4b02-8640-6e5250fe4871 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:18:18.834948 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087d40ff-fdce-4b02-8640-6e5250fe4871 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:18:18.836374 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087d40ff-fdce-4b02-8640-6e5250fe4871 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:18:18.853244 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 480/941] 10.222.0.30 () {56 vars in 1148 bytes} [Tue Sep 6 04:18:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:18:18.853244 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087d40ff-fdce-4b02-8640-6e5250fe4871 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:18:18.885714 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95dc9ba4-091b-4d31-8a9e-0a0b6c62a4d4 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:18:18.893734 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:18:18.919770 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8403611-0ba0-4f08-aa02-8ee76d78fa47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:18:18.920381 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8403611-0ba0-4f08-aa02-8ee76d78fa47 None admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:18:18.920655 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8403611-0ba0-4f08-aa02-8ee76d78fa47 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:18:18.920922 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8403611-0ba0-4f08-aa02-8ee76d78fa47 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:18:18.921622 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8403611-0ba0-4f08-aa02-8ee76d78fa47 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:18:18.922977 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8403611-0ba0-4f08-aa02-8ee76d78fa47 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:18:18.965075 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087d40ff-fdce-4b02-8640-6e5250fe4871 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:18:19.000797 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764b7fed-f6a8-4712-afe2-96c2d760ee76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:18:19.001442 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764b7fed-f6a8-4712-afe2-96c2d760ee76 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:18:19.001981 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764b7fed-f6a8-4712-afe2-96c2d760ee76 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:18:19.002526 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764b7fed-f6a8-4712-afe2-96c2d760ee76 None admin] PATH_INFO: `/v3/services` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:18:19.003143 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-764b7fed-f6a8-4712-afe2-96c2d760ee76 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:18:19.004541 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-764b7fed-f6a8-4712-afe2-96c2d760ee76 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:18:19.010649 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 481/942] 10.222.0.30 () {56 vars in 1087 bytes} [Tue Sep 6 04:18:18 2022] GET /identity/v3/services => generated 1497 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 04:18:19.011388 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764b7fed-f6a8-4712-afe2-96c2d760ee76 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:18:19.042033 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 462/943] 10.222.0.30 () {60 vars in 1144 bytes} [Tue Sep 6 04:18:18 2022] POST /identity/v3/endpoints => generated 347 bytes in 266 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 04:18:19.139972 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:18:19.146324 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764b7fed-f6a8-4712-afe2-96c2d760ee76 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:18:19.204325 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05faf7d-9a4f-4877-8b47-c3e6e4f64771 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:18:19.205413 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05faf7d-9a4f-4877-8b47-c3e6e4f64771 None admin] REQUEST_METHOD: `PUT` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:18:19.205851 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05faf7d-9a4f-4877-8b47-c3e6e4f64771 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:18:19.206307 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05faf7d-9a4f-4877-8b47-c3e6e4f64771 None admin] PATH_INFO: `/v3/projects/b707733ce65c42edbc1fd95ad0a1c06c/users/da5fed17101a4935bcc38b72caf9d263/roles/356631f6a0064bc695b78e3759d55d6a` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:18:19.208978 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05faf7d-9a4f-4877-8b47-c3e6e4f64771 None admin] RBAC: Authorizing `identity:create_grant(project_id=b707733ce65c42edbc1fd95ad0a1c06c, user_id=da5fed17101a4935bcc38b72caf9d263, role_id=356631f6a0064bc695b78e3759d55d6a)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:18:19.210664 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05faf7d-9a4f-4877-8b47-c3e6e4f64771 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:18:19.270290 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 482/944] 10.222.0.30 () {58 vars in 1439 bytes} [Tue Sep 6 04:18:19 2022] PUT /identity/v3/projects/b707733ce65c42edbc1fd95ad0a1c06c/users/da5fed17101a4935bcc38b72caf9d263/roles/356631f6a0064bc695b78e3759d55d6a => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 04:18:19.877860 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8421f3f0-95f5-448a-abec-544c4321b36b None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:18:19.878409 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8421f3f0-95f5-448a-abec-544c4321b36b None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:18:19.878409 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8421f3f0-95f5-448a-abec-544c4321b36b None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:18:19.879905 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 463/945] 10.222.0.30 () {54 vars in 927 bytes} [Tue Sep 6 04:18:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 04:18:19.885130 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff23d28d-343a-4a1f-a3a5-496fb0ecc8d6 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:18:19.885536 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff23d28d-343a-4a1f-a3a5-496fb0ecc8d6 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:18:19.885719 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff23d28d-343a-4a1f-a3a5-496fb0ecc8d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:18:19.903625 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-ff23d28d-343a-4a1f-a3a5-496fb0ecc8d6 None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:18:19.904793 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-ff23d28d-343a-4a1f-a3a5-496fb0ecc8d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:18:19.976131 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 483/946] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Sep 6 04:18:19 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 04:18:20.247195 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8421f3f0-95f5-448a-abec-544c4321b36b None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:18:20.331566 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:18:20.355997 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8421f3f0-95f5-448a-abec-544c4321b36b None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:18:20.396205 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5aff764-9597-4472-b229-ca804f1dd635 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:18:20.396831 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5aff764-9597-4472-b229-ca804f1dd635 None admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:18:20.397128 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5aff764-9597-4472-b229-ca804f1dd635 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:18:20.397417 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5aff764-9597-4472-b229-ca804f1dd635 None admin] PATH_INFO: `/v3/projects` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:18:20.398088 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5aff764-9597-4472-b229-ca804f1dd635 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:18:20.399569 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5aff764-9597-4472-b229-ca804f1dd635 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:18:20.402208 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: WARNING keystone.server.flask.common [None req-a5aff764-9597-4472-b229-ca804f1dd635 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Sep 06 04:18:20.464034 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.sql.core [None req-a5aff764-9597-4472-b229-ca804f1dd635 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 06 04:18:20.464034 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Sep 06 04:18:20.464034 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [parameters: {'id': '9eca9c0811084e81a89ab608bbba66f0', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Sep 06 04:18:20.464034 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81697) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 06 04:18:20.472362 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-a5aff764-9597-4472-b229-ca804f1dd635 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 9eca9c0811084e81a89ab608bbba66f0.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 9eca9c0811084e81a89ab608bbba66f0. Sep 06 04:18:20.472362 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 04:18:20.472362 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 06 04:18:20.472362 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 06 04:18:20.472362 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 06 04:18:20.472362 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 06 04:18:20.472362 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 06 04:18:20.472362 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application result = self._query(query) Sep 06 04:18:20.472362 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 06 04:18:20.472362 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application conn.query(q) Sep 06 04:18:20.472362 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 06 04:18:20.472362 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 06 04:18:20.472362 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 06 04:18:20.473172 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application result.read() Sep 06 04:18:20.473172 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 06 04:18:20.473172 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 06 04:18:20.473172 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 06 04:18:20.473172 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 06 04:18:20.473172 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 06 04:18:20.473172 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 06 04:18:20.473172 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 06 04:18:20.473172 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 06 04:18:20.473172 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 06 04:18:20.473172 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 04:18:20.473172 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 06 04:18:20.473172 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 04:18:20.473172 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 04:18:20.473172 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 06 04:18:20.473172 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 06 04:18:20.473172 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Sep 06 04:18:20.473172 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return project_ref.to_dict() Sep 06 04:18:20.474642 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 06 04:18:20.474642 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application next(self.gen) Sep 06 04:18:20.474642 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 06 04:18:20.474642 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application yield resource Sep 06 04:18:20.474642 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 06 04:18:20.474642 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application next(self.gen) Sep 06 04:18:20.474642 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 06 04:18:20.474642 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application self.session.rollback() Sep 06 04:18:20.474642 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 06 04:18:20.474642 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application self.force_reraise() Sep 06 04:18:20.474642 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 06 04:18:20.474642 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise self.value Sep 06 04:18:20.474642 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 06 04:18:20.474642 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 06 04:18:20.474642 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 06 04:18:20.474642 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application session.commit() Sep 06 04:18:20.474642 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 06 04:18:20.475754 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 06 04:18:20.475754 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 06 04:18:20.475754 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application self._prepare_impl() Sep 06 04:18:20.475754 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 06 04:18:20.475754 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application self.session.flush() Sep 06 04:18:20.475754 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 06 04:18:20.475754 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application self._flush(objects) Sep 06 04:18:20.475754 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 06 04:18:20.475754 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 06 04:18:20.475754 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 06 04:18:20.475754 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application compat.raise_( Sep 06 04:18:20.475754 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 06 04:18:20.475754 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception Sep 06 04:18:20.475754 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 06 04:18:20.475754 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application flush_context.execute() Sep 06 04:18:20.475754 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 06 04:18:20.475754 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rec.execute(self) Sep 06 04:18:20.476656 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 06 04:18:20.476656 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 06 04:18:20.476656 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 06 04:18:20.476656 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 06 04:18:20.476656 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Sep 06 04:18:20.476656 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application c = connection._execute_20( Sep 06 04:18:20.476656 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 06 04:18:20.476656 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 06 04:18:20.476656 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 06 04:18:20.476656 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 06 04:18:20.476656 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 06 04:18:20.476656 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 06 04:18:20.476656 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 06 04:18:20.476656 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 06 04:18:20.476656 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 06 04:18:20.477492 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 06 04:18:20.477492 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 06 04:18:20.477492 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception Sep 06 04:18:20.477492 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 06 04:18:20.477492 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 06 04:18:20.477492 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 06 04:18:20.477492 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 06 04:18:20.477492 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 06 04:18:20.477492 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application result = self._query(query) Sep 06 04:18:20.477492 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 06 04:18:20.477492 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application conn.query(q) Sep 06 04:18:20.477492 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 06 04:18:20.477492 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 06 04:18:20.477492 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 06 04:18:20.477492 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application result.read() Sep 06 04:18:20.477492 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 06 04:18:20.477492 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 06 04:18:20.482870 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 06 04:18:20.482870 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 06 04:18:20.482870 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 06 04:18:20.482870 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 06 04:18:20.482870 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 06 04:18:20.482870 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 06 04:18:20.482870 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 06 04:18:20.482870 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Sep 06 04:18:20.482870 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [parameters: {'id': '9eca9c0811084e81a89ab608bbba66f0', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Sep 06 04:18:20.482870 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 06 04:18:20.482870 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 04:18:20.482870 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 06 04:18:20.482870 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 04:18:20.482870 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 04:18:20.482870 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Sep 06 04:18:20.483870 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Sep 06 04:18:20.483870 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 06 04:18:20.483870 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 06 04:18:20.483870 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Sep 06 04:18:20.483870 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 04:18:20.483870 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 06 04:18:20.483870 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 04:18:20.483870 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 04:18:20.483870 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 04:18:20.483870 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 04:18:20.483870 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 04:18:20.483870 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 04:18:20.483870 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 04:18:20.483870 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 04:18:20.483870 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 04:18:20.483870 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 04:18:20.483870 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 04:18:20.484977 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 04:18:20.484977 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Sep 06 04:18:20.484977 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Sep 06 04:18:20.484977 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 04:18:20.484977 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 04:18:20.484977 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Sep 06 04:18:20.484977 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.Conflict( Sep 06 04:18:20.484977 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 9eca9c0811084e81a89ab608bbba66f0. Sep 06 04:18:20.484977 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 04:18:20.484977 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 464/947] 10.222.0.30 () {60 vars in 1140 bytes} [Tue Sep 6 04:18:20 2022] POST /identity/v3/projects => generated 262 bytes in 227 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Sep 06 04:18:20.485891 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff23d28d-343a-4a1f-a3a5-496fb0ecc8d6 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:18:20.571291 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:18:20.587129 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff23d28d-343a-4a1f-a3a5-496fb0ecc8d6 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:18:20.626518 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90f7c14-f9a0-4a8e-84f9-008314290d10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:18:20.628028 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90f7c14-f9a0-4a8e-84f9-008314290d10 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:18:20.628314 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90f7c14-f9a0-4a8e-84f9-008314290d10 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:18:20.628588 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90f7c14-f9a0-4a8e-84f9-008314290d10 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:18:20.635713 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d90f7c14-f9a0-4a8e-84f9-008314290d10 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:18:20.637169 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d90f7c14-f9a0-4a8e-84f9-008314290d10 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:18:20.648745 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-d90f7c14-f9a0-4a8e-84f9-008314290d10 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 04:18:20.648745 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 04:18:20.648745 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 04:18:20.648745 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 04:18:20.648745 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 04:18:20.648745 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 04:18:20.648745 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 04:18:20.648745 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 04:18:20.648745 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 04:18:20.648745 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 04:18:20.648745 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 04:18:20.648745 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 04:18:20.648745 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 06 04:18:20.648745 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 06 04:18:20.648745 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 06 04:18:20.649718 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 06 04:18:20.649718 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 04:18:20.649718 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 04:18:20.649718 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 04:18:20.649718 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 04:18:20.649718 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 04:18:20.649718 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 04:18:20.649718 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 04:18:20.649718 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 04:18:20.649718 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 04:18:20.649718 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 04:18:20.649718 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 04:18:20.649718 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 04:18:20.649718 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 04:18:20.649718 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 04:18:20.649718 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 04:18:20.649718 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 04:18:20.650590 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 06 04:18:20.650590 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 06 04:18:20.650590 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 06 04:18:20.650590 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 06 04:18:20.650590 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 06 04:18:20.650590 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 06 04:18:20.650590 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 04:18:20.650590 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 04:18:20.650590 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 484/948] 10.222.0.30 () {56 vars in 1111 bytes} [Tue Sep 6 04:18:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 04:18:20.659490 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5aff764-9597-4472-b229-ca804f1dd635 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:18:20.772711 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5aff764-9597-4472-b229-ca804f1dd635 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:18:20.805051 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:18:20.810413 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bfc9eff-0f6b-41f3-8608-84625bc0f628 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:18:20.811037 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfc9eff-0f6b-41f3-8608-84625bc0f628 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:18:20.811321 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfc9eff-0f6b-41f3-8608-84625bc0f628 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:18:20.811594 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfc9eff-0f6b-41f3-8608-84625bc0f628 None admin] PATH_INFO: `/v3/projects` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:18:20.812268 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bfc9eff-0f6b-41f3-8608-84625bc0f628 None admin] RBAC: Adding query filter params (name=service) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:18:20.812571 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bfc9eff-0f6b-41f3-8608-84625bc0f628 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:18:20.814232 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bfc9eff-0f6b-41f3-8608-84625bc0f628 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:18:20.837955 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 465/949] 10.222.0.30 () {56 vars in 1112 bytes} [Tue Sep 6 04:18:20 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:18:21.028219 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623509b0-a8ef-4f81-a169-9d04e7f784fc None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:18:21.028742 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623509b0-a8ef-4f81-a169-9d04e7f784fc None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:18:21.028742 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623509b0-a8ef-4f81-a169-9d04e7f784fc None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:18:21.030440 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 485/950] 10.222.0.30 () {54 vars in 927 bytes} [Tue Sep 6 04:18:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 04:18:21.035910 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8e3420-6c98-4115-8d0a-8fa769a8df92 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:18:21.036331 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8e3420-6c98-4115-8d0a-8fa769a8df92 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:18:21.036525 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8e3420-6c98-4115-8d0a-8fa769a8df92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:18:21.058105 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-7e8e3420-6c98-4115-8d0a-8fa769a8df92 None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:18:21.059640 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-7e8e3420-6c98-4115-8d0a-8fa769a8df92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:18:21.132914 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 466/951] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Sep 6 04:18:21 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 04:18:21.208039 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6806b9a-c266-409c-9660-de7ae55a852c None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:18:21.208467 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6806b9a-c266-409c-9660-de7ae55a852c None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:18:21.208870 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6806b9a-c266-409c-9660-de7ae55a852c None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:18:21.211446 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 486/952] 10.222.0.30 () {54 vars in 927 bytes} [Tue Sep 6 04:18:21 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 04:18:21.219136 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cd45eb-b855-4b53-a310-1f77838fd1a8 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:18:21.219567 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cd45eb-b855-4b53-a310-1f77838fd1a8 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:18:21.219982 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cd45eb-b855-4b53-a310-1f77838fd1a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:18:21.244273 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-36cd45eb-b855-4b53-a310-1f77838fd1a8 None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:18:21.245491 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-36cd45eb-b855-4b53-a310-1f77838fd1a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:18:21.317911 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 467/953] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Sep 6 04:18:21 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 04:18:21.344460 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0024463-46c1-4d2b-a5a0-62b4783eeeeb None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:18:21.344776 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0024463-46c1-4d2b-a5a0-62b4783eeeeb None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:18:21.345009 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0024463-46c1-4d2b-a5a0-62b4783eeeeb None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:18:21.347593 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 487/954] 10.222.0.30 () {54 vars in 927 bytes} [Tue Sep 6 04:18:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 04:18:21.352706 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e722c1bf-e9b6-4ff1-9275-16b901301826 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:18:21.352998 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e722c1bf-e9b6-4ff1-9275-16b901301826 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:18:21.353273 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e722c1bf-e9b6-4ff1-9275-16b901301826 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:18:21.373618 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-e722c1bf-e9b6-4ff1-9275-16b901301826 None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:18:21.374817 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-e722c1bf-e9b6-4ff1-9275-16b901301826 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:18:21.413300 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0024463-46c1-4d2b-a5a0-62b4783eeeeb None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:18:21.433667 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:18:21.458948 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 468/955] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Sep 6 04:18:21 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 04:18:21.531089 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0024463-46c1-4d2b-a5a0-62b4783eeeeb None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:18:21.567623 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5abb90-fb57-42d2-a95b-cf781efe8e50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:18:21.568562 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5abb90-fb57-42d2-a95b-cf781efe8e50 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:18:21.569026 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5abb90-fb57-42d2-a95b-cf781efe8e50 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:18:21.569506 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5abb90-fb57-42d2-a95b-cf781efe8e50 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:18:21.570249 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5abb90-fb57-42d2-a95b-cf781efe8e50 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:18:21.571852 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5abb90-fb57-42d2-a95b-cf781efe8e50 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:18:21.582489 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-eb5abb90-fb57-42d2-a95b-cf781efe8e50 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 06 04:18:21.582489 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 04:18:21.582489 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 06 04:18:21.582489 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 06 04:18:21.582489 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 06 04:18:21.582489 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 06 04:18:21.582489 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 06 04:18:21.582489 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 04:18:21.582489 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 06 04:18:21.582489 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 04:18:21.582489 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 06 04:18:21.582489 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 04:18:21.582489 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 04:18:21.582489 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 04:18:21.582489 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 04:18:21.582489 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 04:18:21.582489 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 04:18:21.583884 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 04:18:21.583884 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 04:18:21.583884 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 04:18:21.583884 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 04:18:21.583884 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 04:18:21.583884 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 04:18:21.583884 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 06 04:18:21.583884 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 06 04:18:21.583884 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 06 04:18:21.583884 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 06 04:18:21.583884 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 04:18:21.583884 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 04:18:21.583884 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 04:18:21.583884 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 04:18:21.583884 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 04:18:21.583884 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 04:18:21.585142 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 04:18:21.585142 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 04:18:21.585142 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 04:18:21.585142 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 04:18:21.585142 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 04:18:21.585142 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 04:18:21.585142 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 04:18:21.585142 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 04:18:21.585142 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 04:18:21.585142 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 04:18:21.585142 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 06 04:18:21.585142 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 04:18:21.585142 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 06 04:18:21.585142 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 04:18:21.595242 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 488/956] 10.222.0.30 () {56 vars in 1126 bytes} [Tue Sep 6 04:18:21 2022] GET /identity/v3/services/object-store => generated 93 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 04:18:21.598792 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e722c1bf-e9b6-4ff1-9275-16b901301826 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:18:21.609202 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5abb90-fb57-42d2-a95b-cf781efe8e50 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:18:21.627790 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:18:21.732175 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e722c1bf-e9b6-4ff1-9275-16b901301826 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:18:21.733553 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5abb90-fb57-42d2-a95b-cf781efe8e50 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:18:21.770148 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aadda44-4e1b-4294-af7d-8eaa7cd924fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:18:21.770573 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c9c603-d738-449d-bc46-7cbe9865900a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:18:21.770795 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aadda44-4e1b-4294-af7d-8eaa7cd924fe None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:18:21.771048 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aadda44-4e1b-4294-af7d-8eaa7cd924fe None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:18:21.771318 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aadda44-4e1b-4294-af7d-8eaa7cd924fe None admin] PATH_INFO: `/v3/services/compute` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:18:21.771895 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aadda44-4e1b-4294-af7d-8eaa7cd924fe None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:18:21.772223 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c9c603-d738-449d-bc46-7cbe9865900a None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:18:21.772710 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c9c603-d738-449d-bc46-7cbe9865900a None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:18:21.773195 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c9c603-d738-449d-bc46-7cbe9865900a None admin] PATH_INFO: `/v3/services` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:18:21.773345 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aadda44-4e1b-4294-af7d-8eaa7cd924fe None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:18:21.774348 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c9c603-d738-449d-bc46-7cbe9865900a None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:18:21.774883 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c9c603-d738-449d-bc46-7cbe9865900a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:18:21.776384 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c9c603-d738-449d-bc46-7cbe9865900a None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:18:21.781745 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 469/957] 10.222.0.30 () {56 vars in 1122 bytes} [Tue Sep 6 04:18:21 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:18:21.782103 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-6aadda44-4e1b-4294-af7d-8eaa7cd924fe None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 06 04:18:21.782103 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 04:18:21.782103 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 06 04:18:21.782103 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 06 04:18:21.782103 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 06 04:18:21.782103 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 06 04:18:21.782103 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 06 04:18:21.782103 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 04:18:21.782103 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 06 04:18:21.782103 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 04:18:21.782103 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 06 04:18:21.782103 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 04:18:21.782103 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 04:18:21.782103 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 04:18:21.782103 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 04:18:21.782103 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 04:18:21.782103 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 04:18:21.783128 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 04:18:21.783128 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 04:18:21.783128 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 04:18:21.783128 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 04:18:21.783128 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 04:18:21.783128 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 04:18:21.783128 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 06 04:18:21.783128 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 06 04:18:21.783128 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 06 04:18:21.783128 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 06 04:18:21.783128 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 04:18:21.783128 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 04:18:21.783128 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 04:18:21.783128 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 04:18:21.783128 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 04:18:21.783128 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 04:18:21.784964 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 04:18:21.784964 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 04:18:21.784964 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 04:18:21.784964 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 04:18:21.784964 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 04:18:21.784964 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 04:18:21.784964 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 04:18:21.784964 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 04:18:21.784964 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 04:18:21.784964 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 04:18:21.784964 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 06 04:18:21.784964 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 04:18:21.784964 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 06 04:18:21.784964 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 04:18:21.786463 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c9c603-d738-449d-bc46-7cbe9865900a None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:18:21.788539 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 489/958] 10.222.0.30 () {56 vars in 1111 bytes} [Tue Sep 6 04:18:21 2022] GET /identity/v3/services/compute => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 04:18:21.789440 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aadda44-4e1b-4294-af7d-8eaa7cd924fe None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:18:21.837710 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:18:21.868390 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:18:21.894981 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aadda44-4e1b-4294-af7d-8eaa7cd924fe None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:18:21.900144 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c9c603-d738-449d-bc46-7cbe9865900a None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:18:21.931585 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e90779-3467-4b72-abc7-977a2c8f6a84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:18:21.932819 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e90779-3467-4b72-abc7-977a2c8f6a84 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:18:21.932819 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e90779-3467-4b72-abc7-977a2c8f6a84 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:18:21.932819 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e90779-3467-4b72-abc7-977a2c8f6a84 None admin] PATH_INFO: `/v3/services` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:18:21.933385 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e90779-3467-4b72-abc7-977a2c8f6a84 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:18:21.933694 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e90779-3467-4b72-abc7-977a2c8f6a84 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:18:21.935063 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e90779-3467-4b72-abc7-977a2c8f6a84 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:18:21.939976 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae9eb5e-79de-4127-8ee1-87b1c0edbc6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:18:21.940642 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 490/959] 10.222.0.30 () {56 vars in 1122 bytes} [Tue Sep 6 04:18:21 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:18:21.941355 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae9eb5e-79de-4127-8ee1-87b1c0edbc6d None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:18:21.941573 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e90779-3467-4b72-abc7-977a2c8f6a84 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:18:21.941988 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae9eb5e-79de-4127-8ee1-87b1c0edbc6d None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:18:21.942264 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae9eb5e-79de-4127-8ee1-87b1c0edbc6d None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:18:21.957469 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae9eb5e-79de-4127-8ee1-87b1c0edbc6d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 04:18:21.962449 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-0ae9eb5e-79de-4127-8ee1-87b1c0edbc6d None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 06 04:18:21.962449 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 04:18:21.962449 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 04:18:21.962449 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 04:18:21.962449 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 04:18:21.962449 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 04:18:21.962449 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 04:18:21.962449 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 04:18:21.962449 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 04:18:21.962449 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 04:18:21.962449 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 04:18:21.962449 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 04:18:21.962449 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 04:18:21.962449 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 04:18:21.962449 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 04:18:21.962449 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 04:18:21.963411 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 04:18:21.963411 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 04:18:21.963411 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 04:18:21.963411 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 04:18:21.963411 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 04:18:21.963411 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 04:18:21.963411 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 04:18:21.963411 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 04:18:21.963411 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 04:18:21.963411 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 04:18:21.963411 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 04:18:21.963411 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 04:18:21.963411 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 04:18:21.963411 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 04:18:21.963411 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 04:18:21.963411 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 04:18:21.964294 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 04:18:21.964294 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 04:18:21.964294 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 04:18:21.964294 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 04:18:21.964294 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 04:18:21.964294 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 04:18:21.964294 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 04:18:21.964294 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 04:18:21.964294 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 04:18:21.964294 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 04:18:21.964294 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 06 04:18:21.964294 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 04:18:21.964294 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 470/960] 10.222.0.30 () {56 vars in 1120 bytes} [Tue Sep 6 04:18:21 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 04:18:21.965034 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae9eb5e-79de-4127-8ee1-87b1c0edbc6d None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:18:22.072201 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e90779-3467-4b72-abc7-977a2c8f6a84 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:18:22.089266 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae9eb5e-79de-4127-8ee1-87b1c0edbc6d None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:18:22.097552 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:18:22.109965 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617dfc99-f279-4f94-93f5-20dae7f5b0c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:18:22.110646 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617dfc99-f279-4f94-93f5-20dae7f5b0c0 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:18:22.110930 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617dfc99-f279-4f94-93f5-20dae7f5b0c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:18:22.113178 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617dfc99-f279-4f94-93f5-20dae7f5b0c0 None admin] PATH_INFO: `/v3/services` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:18:22.113178 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617dfc99-f279-4f94-93f5-20dae7f5b0c0 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:18:22.113178 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617dfc99-f279-4f94-93f5-20dae7f5b0c0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:18:22.113747 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617dfc99-f279-4f94-93f5-20dae7f5b0c0 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:18:22.123370 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 491/961] 10.222.0.30 () {56 vars in 1112 bytes} [Tue Sep 6 04:18:21 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:18:22.123370 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617dfc99-f279-4f94-93f5-20dae7f5b0c0 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:18:22.123370 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:18:22.131059 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d39b63-56ac-4346-8cd1-296a068f5dab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:18:22.131741 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d39b63-56ac-4346-8cd1-296a068f5dab None admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:18:22.132378 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d39b63-56ac-4346-8cd1-296a068f5dab None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:18:22.132989 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d39b63-56ac-4346-8cd1-296a068f5dab None admin] PATH_INFO: `/v3/endpoints` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:18:22.134195 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d39b63-56ac-4346-8cd1-296a068f5dab None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:18:22.135653 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d39b63-56ac-4346-8cd1-296a068f5dab None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:18:22.213420 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 471/962] 10.222.0.30 () {60 vars in 1144 bytes} [Tue Sep 6 04:18:21 2022] POST /identity/v3/endpoints => generated 349 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 04:18:22.214617 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d39b63-56ac-4346-8cd1-296a068f5dab None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:18:22.233510 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617dfc99-f279-4f94-93f5-20dae7f5b0c0 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:18:22.282168 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc91a85-dc28-4762-b997-3d2114a81e2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:18:22.282858 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc91a85-dc28-4762-b997-3d2114a81e2e None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:18:22.283760 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc91a85-dc28-4762-b997-3d2114a81e2e None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:18:22.284338 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc91a85-dc28-4762-b997-3d2114a81e2e None admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:18:22.285032 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc91a85-dc28-4762-b997-3d2114a81e2e None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:18:22.285354 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc91a85-dc28-4762-b997-3d2114a81e2e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:18:22.286931 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc91a85-dc28-4762-b997-3d2114a81e2e None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:18:22.297509 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 492/963] 10.222.0.30 () {56 vars in 1115 bytes} [Tue Sep 6 04:18:22 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:18:22.306587 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc91a85-dc28-4762-b997-3d2114a81e2e None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:18:22.373862 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d39b63-56ac-4346-8cd1-296a068f5dab None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:18:22.390011 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:18:22.428366 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b059af-666b-48f8-9cf5-05d5fb0fd57d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:18:22.428994 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b059af-666b-48f8-9cf5-05d5fb0fd57d None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:18:22.429279 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b059af-666b-48f8-9cf5-05d5fb0fd57d None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:18:22.429588 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b059af-666b-48f8-9cf5-05d5fb0fd57d None admin] PATH_INFO: `/v3/services` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:18:22.430233 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b059af-666b-48f8-9cf5-05d5fb0fd57d None admin] RBAC: Adding query filter params (type=compute) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:18:22.430537 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b059af-666b-48f8-9cf5-05d5fb0fd57d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:18:22.431946 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b059af-666b-48f8-9cf5-05d5fb0fd57d None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:18:22.444785 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 472/964] 10.222.0.30 () {56 vars in 1112 bytes} [Tue Sep 6 04:18:22 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:18:22.456923 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc91a85-dc28-4762-b997-3d2114a81e2e None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:18:22.498678 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0b1ba2-78f9-48a1-81ad-3dc49371fbcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:18:22.499344 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0b1ba2-78f9-48a1-81ad-3dc49371fbcb None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:18:22.499628 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0b1ba2-78f9-48a1-81ad-3dc49371fbcb None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:18:22.499901 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0b1ba2-78f9-48a1-81ad-3dc49371fbcb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:18:22.513153 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f0b1ba2-78f9-48a1-81ad-3dc49371fbcb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:18:22.514711 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f0b1ba2-78f9-48a1-81ad-3dc49371fbcb None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:18:22.531778 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-5f0b1ba2-78f9-48a1-81ad-3dc49371fbcb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 04:18:22.531778 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 04:18:22.531778 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 04:18:22.531778 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 04:18:22.531778 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 04:18:22.531778 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 04:18:22.531778 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 04:18:22.531778 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 04:18:22.531778 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 04:18:22.531778 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 04:18:22.531778 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 04:18:22.531778 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 04:18:22.531778 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 04:18:22.531778 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 04:18:22.531778 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 04:18:22.531778 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 04:18:22.532814 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 04:18:22.532814 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 04:18:22.532814 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 04:18:22.532814 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 04:18:22.532814 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 04:18:22.532814 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 04:18:22.532814 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 04:18:22.532814 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 04:18:22.532814 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 04:18:22.532814 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 04:18:22.532814 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 04:18:22.532814 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 04:18:22.532814 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 04:18:22.532814 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 04:18:22.532814 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 04:18:22.532814 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 04:18:22.532814 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 04:18:22.534515 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 04:18:22.534515 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 04:18:22.534515 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 04:18:22.537409 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 493/965] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Sep 6 04:18:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 04:18:22.539433 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b059af-666b-48f8-9cf5-05d5fb0fd57d None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:18:22.572895 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:18:22.656741 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b059af-666b-48f8-9cf5-05d5fb0fd57d None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:18:22.693978 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435f3b89-acd8-44d5-976f-8d8897b7a7bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:18:22.694986 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435f3b89-acd8-44d5-976f-8d8897b7a7bd None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:18:22.695462 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435f3b89-acd8-44d5-976f-8d8897b7a7bd None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:18:22.695929 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435f3b89-acd8-44d5-976f-8d8897b7a7bd None admin] PATH_INFO: `/v3/domains` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:18:22.696765 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435f3b89-acd8-44d5-976f-8d8897b7a7bd None admin] RBAC: Adding query filter params (name=Default) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:18:22.697261 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435f3b89-acd8-44d5-976f-8d8897b7a7bd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:18:22.698845 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435f3b89-acd8-44d5-976f-8d8897b7a7bd None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:18:22.713862 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 473/966] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Sep 6 04:18:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:18:22.722865 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0b1ba2-78f9-48a1-81ad-3dc49371fbcb None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:18:22.827607 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0b1ba2-78f9-48a1-81ad-3dc49371fbcb None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:18:22.841540 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:18:22.863821 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687e07ad-c4c0-405d-862a-95b49d4b11be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:18:22.864805 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687e07ad-c4c0-405d-862a-95b49d4b11be None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:18:22.865419 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687e07ad-c4c0-405d-862a-95b49d4b11be None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:18:22.865905 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687e07ad-c4c0-405d-862a-95b49d4b11be None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:18:22.874153 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687e07ad-c4c0-405d-862a-95b49d4b11be None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:18:22.875864 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687e07ad-c4c0-405d-862a-95b49d4b11be None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:18:22.888959 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-687e07ad-c4c0-405d-862a-95b49d4b11be None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 06 04:18:22.888959 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 04:18:22.888959 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 04:18:22.888959 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 04:18:22.888959 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 04:18:22.888959 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 04:18:22.888959 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 04:18:22.888959 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 04:18:22.888959 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 04:18:22.888959 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 04:18:22.888959 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 04:18:22.888959 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 04:18:22.888959 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 06 04:18:22.888959 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 06 04:18:22.888959 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 06 04:18:22.888959 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 06 04:18:22.889959 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 04:18:22.889959 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 04:18:22.889959 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 06 04:18:22.889959 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 04:18:22.889959 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 06 04:18:22.889959 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 04:18:22.889959 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 04:18:22.889959 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 04:18:22.889959 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 04:18:22.889959 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 04:18:22.889959 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 04:18:22.889959 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 04:18:22.889959 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 04:18:22.889959 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 04:18:22.889959 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 04:18:22.889959 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 04:18:22.889959 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 04:18:22.889959 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 04:18:22.890895 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 04:18:22.890895 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 04:18:22.890895 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 06 04:18:22.890895 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 06 04:18:22.890895 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 06 04:18:22.890895 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 06 04:18:22.890895 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 06 04:18:22.890895 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 06 04:18:22.890895 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 06 04:18:22.890895 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 04:18:22.893831 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 494/967] 10.222.0.30 () {56 vars in 1117 bytes} [Tue Sep 6 04:18:22 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 04:18:22.904545 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435f3b89-acd8-44d5-976f-8d8897b7a7bd None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:18:22.964888 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72857368-8db4-44dc-9280-3263225da5f1 None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:18:22.965212 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72857368-8db4-44dc-9280-3263225da5f1 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:18:22.965455 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72857368-8db4-44dc-9280-3263225da5f1 None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:18:22.966804 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 495/968] 10.222.0.30 () {54 vars in 927 bytes} [Tue Sep 6 04:18:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 04:18:22.973266 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a84ab01-8476-4563-b51d-344896bef6cf None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:18:22.973611 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a84ab01-8476-4563-b51d-344896bef6cf None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:18:22.973877 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a84ab01-8476-4563-b51d-344896bef6cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:18:22.996828 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:18:23.009073 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-3a84ab01-8476-4563-b51d-344896bef6cf None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:18:23.011047 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-3a84ab01-8476-4563-b51d-344896bef6cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:18:23.014212 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435f3b89-acd8-44d5-976f-8d8897b7a7bd None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:18:23.051724 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21031c3-b906-49ef-a3eb-446bc6a2f7bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:18:23.052580 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21031c3-b906-49ef-a3eb-446bc6a2f7bb None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:18:23.053011 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21031c3-b906-49ef-a3eb-446bc6a2f7bb None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:18:23.053486 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21031c3-b906-49ef-a3eb-446bc6a2f7bb None admin] PATH_INFO: `/v3/users` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:18:23.054549 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21031c3-b906-49ef-a3eb-446bc6a2f7bb None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:18:23.055101 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21031c3-b906-49ef-a3eb-446bc6a2f7bb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:18:23.056704 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21031c3-b906-49ef-a3eb-446bc6a2f7bb None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:18:23.080205 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 474/969] 10.222.0.30 () {56 vars in 1149 bytes} [Tue Sep 6 04:18:22 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:18:23.081441 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21031c3-b906-49ef-a3eb-446bc6a2f7bb None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:18:23.109726 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 496/970] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Sep 6 04:18:22 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 06 04:18:23.191052 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21031c3-b906-49ef-a3eb-446bc6a2f7bb None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:18:23.231190 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c24f11-e397-4434-8b44-ff9a64f5814c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:18:23.231730 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c24f11-e397-4434-8b44-ff9a64f5814c None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:18:23.232015 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c24f11-e397-4434-8b44-ff9a64f5814c None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:18:23.232294 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c24f11-e397-4434-8b44-ff9a64f5814c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:18:23.250111 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:18:23.310268 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c24f11-e397-4434-8b44-ff9a64f5814c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:18:23.312121 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c24f11-e397-4434-8b44-ff9a64f5814c None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:18:23.414807 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a84ab01-8476-4563-b51d-344896bef6cf None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:18:23.418831 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 475/971] 10.222.0.30 () {58 vars in 1282 bytes} [Tue Sep 6 04:18:23 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 06 04:18:23.426253 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c24f11-e397-4434-8b44-ff9a64f5814c None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:18:23.477107 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:18:23.532484 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a84ab01-8476-4563-b51d-344896bef6cf None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:18:23.534655 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:18:23.542117 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c24f11-e397-4434-8b44-ff9a64f5814c None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:18:23.572374 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445e076a-d961-489e-9ffd-5432e5436b0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:18:23.572997 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445e076a-d961-489e-9ffd-5432e5436b0d None admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:18:23.573279 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445e076a-d961-489e-9ffd-5432e5436b0d None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:18:23.573582 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445e076a-d961-489e-9ffd-5432e5436b0d None admin] PATH_INFO: `/v3/users` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:18:23.574227 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445e076a-d961-489e-9ffd-5432e5436b0d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:18:23.575881 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445e076a-d961-489e-9ffd-5432e5436b0d None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:18:23.578776 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: WARNING keystone.server.flask.common [None req-445e076a-d961-489e-9ffd-5432e5436b0d None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Sep 06 04:18:23.580955 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da3ed57-cce6-42e2-94f1-8eb5e512d663 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:18:23.581838 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da3ed57-cce6-42e2-94f1-8eb5e512d663 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:18:23.582300 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da3ed57-cce6-42e2-94f1-8eb5e512d663 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:18:23.582770 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da3ed57-cce6-42e2-94f1-8eb5e512d663 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:18:23.593932 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da3ed57-cce6-42e2-94f1-8eb5e512d663 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:18:23.596210 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da3ed57-cce6-42e2-94f1-8eb5e512d663 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:18:23.612051 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-5da3ed57-cce6-42e2-94f1-8eb5e512d663 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 04:18:23.612051 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 04:18:23.612051 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 04:18:23.612051 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 04:18:23.612051 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 04:18:23.612051 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 04:18:23.612051 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 04:18:23.612051 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 04:18:23.612051 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 04:18:23.612051 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 04:18:23.612051 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 04:18:23.612051 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 04:18:23.612051 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 04:18:23.612051 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 04:18:23.612051 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 04:18:23.612051 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 04:18:23.613018 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 04:18:23.613018 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 04:18:23.613018 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 04:18:23.613018 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 04:18:23.613018 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 04:18:23.613018 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 04:18:23.613018 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 04:18:23.613018 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 04:18:23.613018 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 04:18:23.613018 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 04:18:23.613018 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 04:18:23.613018 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 04:18:23.613018 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 04:18:23.613018 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 04:18:23.613018 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 04:18:23.613018 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 04:18:23.613018 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 04:18:23.614287 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 04:18:23.614287 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 04:18:23.614287 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 04:18:23.614653 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 476/972] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Sep 6 04:18:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 04:18:23.619598 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da3ed57-cce6-42e2-94f1-8eb5e512d663 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:18:23.626566 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.sql.core [None req-445e076a-d961-489e-9ffd-5432e5436b0d None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 06 04:18:23.626566 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Sep 06 04:18:23.626566 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [parameters: {'user_id': '175a0081594b43bab7f9f523d44ebcc6', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Sep 06 04:18:23.626566 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81698) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 06 04:18:23.651550 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-445e076a-d961-489e-9ffd-5432e5436b0d None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Sep 06 04:18:23.651550 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 04:18:23.651550 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 06 04:18:23.651550 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 06 04:18:23.651550 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 06 04:18:23.651550 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 06 04:18:23.651550 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 06 04:18:23.651550 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application result = self._query(query) Sep 06 04:18:23.651550 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 06 04:18:23.651550 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application conn.query(q) Sep 06 04:18:23.651550 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 06 04:18:23.651550 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 06 04:18:23.651550 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 06 04:18:23.651550 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application result.read() Sep 06 04:18:23.651550 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 06 04:18:23.652554 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 06 04:18:23.652554 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 06 04:18:23.652554 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 06 04:18:23.652554 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 06 04:18:23.652554 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 06 04:18:23.652554 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 06 04:18:23.652554 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 06 04:18:23.652554 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 06 04:18:23.652554 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 04:18:23.652554 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 06 04:18:23.652554 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 04:18:23.652554 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 04:18:23.652554 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 06 04:18:23.652554 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 06 04:18:23.652554 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Sep 06 04:18:23.652554 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Sep 06 04:18:23.652554 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 06 04:18:23.652554 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application next(self.gen) Sep 06 04:18:23.653664 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 06 04:18:23.653664 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application yield resource Sep 06 04:18:23.653664 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 06 04:18:23.653664 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application next(self.gen) Sep 06 04:18:23.653664 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 06 04:18:23.653664 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application self.session.rollback() Sep 06 04:18:23.653664 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 06 04:18:23.653664 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application self.force_reraise() Sep 06 04:18:23.653664 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 06 04:18:23.653664 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise self.value Sep 06 04:18:23.653664 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 06 04:18:23.653664 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 06 04:18:23.653664 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 06 04:18:23.653664 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application session.commit() Sep 06 04:18:23.653664 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 06 04:18:23.653664 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 06 04:18:23.653664 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 06 04:18:23.654578 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application self._prepare_impl() Sep 06 04:18:23.654578 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 06 04:18:23.654578 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application self.session.flush() Sep 06 04:18:23.654578 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 06 04:18:23.654578 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application self._flush(objects) Sep 06 04:18:23.654578 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 06 04:18:23.654578 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 06 04:18:23.654578 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 06 04:18:23.654578 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application compat.raise_( Sep 06 04:18:23.654578 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 06 04:18:23.654578 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception Sep 06 04:18:23.654578 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 06 04:18:23.654578 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application flush_context.execute() Sep 06 04:18:23.654578 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 06 04:18:23.654578 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rec.execute(self) Sep 06 04:18:23.654578 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 06 04:18:23.654578 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 06 04:18:23.655510 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 06 04:18:23.655510 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 06 04:18:23.655510 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Sep 06 04:18:23.655510 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application result = connection._execute_20( Sep 06 04:18:23.655510 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 06 04:18:23.655510 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 06 04:18:23.655510 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 06 04:18:23.655510 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 06 04:18:23.655510 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 06 04:18:23.655510 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 06 04:18:23.655510 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 06 04:18:23.655510 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 06 04:18:23.655510 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 06 04:18:23.655510 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 06 04:18:23.655510 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 06 04:18:23.655510 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception Sep 06 04:18:23.658522 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 06 04:18:23.658522 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 06 04:18:23.658522 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 06 04:18:23.658522 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 06 04:18:23.658522 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 06 04:18:23.658522 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application result = self._query(query) Sep 06 04:18:23.658522 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 06 04:18:23.658522 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application conn.query(q) Sep 06 04:18:23.658522 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 06 04:18:23.658522 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 06 04:18:23.658522 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 06 04:18:23.658522 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application result.read() Sep 06 04:18:23.658522 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 06 04:18:23.658522 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 06 04:18:23.658522 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 06 04:18:23.658522 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 06 04:18:23.660906 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 06 04:18:23.660906 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 06 04:18:23.660906 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 06 04:18:23.660906 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 06 04:18:23.660906 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 06 04:18:23.660906 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Sep 06 04:18:23.660906 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [parameters: {'user_id': '175a0081594b43bab7f9f523d44ebcc6', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Sep 06 04:18:23.660906 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 06 04:18:23.660906 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 04:18:23.660906 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 06 04:18:23.660906 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 04:18:23.660906 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 04:18:23.660906 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 04:18:23.660906 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 04:18:23.660906 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 04:18:23.663278 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 04:18:23.663278 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 04:18:23.663278 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 04:18:23.663278 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 04:18:23.663278 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 04:18:23.663278 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 04:18:23.663278 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 04:18:23.663278 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Sep 06 04:18:23.663278 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Sep 06 04:18:23.663278 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 04:18:23.663278 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 04:18:23.663278 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 06 04:18:23.663278 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 04:18:23.663278 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 06 04:18:23.663278 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 04:18:23.663278 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Sep 06 04:18:23.663278 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Sep 06 04:18:23.665853 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Sep 06 04:18:23.665853 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Sep 06 04:18:23.665853 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 06 04:18:23.665853 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 06 04:18:23.665853 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Sep 06 04:18:23.665853 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 04:18:23.665853 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 497/973] 10.222.0.30 () {60 vars in 1131 bytes} [Tue Sep 6 04:18:23 2022] POST /identity/v3/users => generated 160 bytes in 249 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Sep 06 04:18:23.666752 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445e076a-d961-489e-9ffd-5432e5436b0d None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:18:23.761579 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:18:23.770689 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da3ed57-cce6-42e2-94f1-8eb5e512d663 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:18:23.783212 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445e076a-d961-489e-9ffd-5432e5436b0d None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:18:23.819516 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e796e0-feaa-47dd-bf8b-42a2c92381f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:18:23.820708 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e796e0-feaa-47dd-bf8b-42a2c92381f0 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:18:23.821277 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e796e0-feaa-47dd-bf8b-42a2c92381f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:18:23.821863 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e796e0-feaa-47dd-bf8b-42a2c92381f0 None admin] PATH_INFO: `/v3/domains` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:18:23.822758 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e796e0-feaa-47dd-bf8b-42a2c92381f0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:18:23.823239 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e796e0-feaa-47dd-bf8b-42a2c92381f0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:18:23.824176 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabd1ef0-63fe-48a9-83f3-fafa6a4b45de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:18:23.824906 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabd1ef0-63fe-48a9-83f3-fafa6a4b45de None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:18:23.825258 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e796e0-feaa-47dd-bf8b-42a2c92381f0 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:18:23.825402 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabd1ef0-63fe-48a9-83f3-fafa6a4b45de None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:18:23.825735 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabd1ef0-63fe-48a9-83f3-fafa6a4b45de None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:18:23.833542 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cabd1ef0-63fe-48a9-83f3-fafa6a4b45de None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:18:23.835394 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cabd1ef0-63fe-48a9-83f3-fafa6a4b45de None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:18:23.841954 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 477/974] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Sep 6 04:18:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 04:18:23.847712 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e796e0-feaa-47dd-bf8b-42a2c92381f0 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:18:23.853121 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-cabd1ef0-63fe-48a9-83f3-fafa6a4b45de None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 06 04:18:23.853121 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 04:18:23.853121 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 04:18:23.853121 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 04:18:23.853121 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 04:18:23.853121 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 04:18:23.853121 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 04:18:23.853121 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 04:18:23.853121 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 04:18:23.853121 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 04:18:23.853121 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 04:18:23.853121 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 04:18:23.853121 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 06 04:18:23.853121 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 06 04:18:23.853121 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 06 04:18:23.853121 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 06 04:18:23.854305 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 04:18:23.854305 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 04:18:23.854305 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 06 04:18:23.854305 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 04:18:23.854305 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 06 04:18:23.854305 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 04:18:23.854305 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 04:18:23.854305 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 04:18:23.854305 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 04:18:23.854305 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 04:18:23.854305 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 04:18:23.854305 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 04:18:23.854305 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 04:18:23.854305 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 04:18:23.854305 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 04:18:23.854305 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 04:18:23.854305 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 04:18:23.854305 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 04:18:23.855498 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 04:18:23.855498 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 04:18:23.855498 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 06 04:18:23.855498 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 06 04:18:23.855498 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 06 04:18:23.855498 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 06 04:18:23.855498 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 06 04:18:23.855498 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 06 04:18:23.855498 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 06 04:18:23.855498 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 04:18:23.857692 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 498/975] 10.222.0.30 () {56 vars in 1099 bytes} [Tue Sep 6 04:18:23 2022] GET /identity/v3/users/cinder => generated 84 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 04:18:23.863969 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabd1ef0-63fe-48a9-83f3-fafa6a4b45de None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:18:23.892014 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:18:23.981281 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e796e0-feaa-47dd-bf8b-42a2c92381f0 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:18:23.987219 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabd1ef0-63fe-48a9-83f3-fafa6a4b45de None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:18:24.023695 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497244bd-8723-447e-b28f-304d60b204e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:18:24.024383 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497244bd-8723-447e-b28f-304d60b204e7 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:18:24.024622 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497244bd-8723-447e-b28f-304d60b204e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:18:24.024954 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497244bd-8723-447e-b28f-304d60b204e7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:18:24.034834 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497244bd-8723-447e-b28f-304d60b204e7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:18:24.036480 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497244bd-8723-447e-b28f-304d60b204e7 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:18:24.039625 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69720ecc-5ed6-44d4-82fd-bf842ea880f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:18:24.040503 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69720ecc-5ed6-44d4-82fd-bf842ea880f3 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:18:24.040999 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69720ecc-5ed6-44d4-82fd-bf842ea880f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:18:24.041652 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69720ecc-5ed6-44d4-82fd-bf842ea880f3 None admin] PATH_INFO: `/v3/users` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:18:24.042630 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69720ecc-5ed6-44d4-82fd-bf842ea880f3 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:18:24.043383 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69720ecc-5ed6-44d4-82fd-bf842ea880f3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:18:24.045270 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69720ecc-5ed6-44d4-82fd-bf842ea880f3 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:18:24.050385 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-497244bd-8723-447e-b28f-304d60b204e7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 04:18:24.050385 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 04:18:24.050385 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 04:18:24.050385 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 04:18:24.050385 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 04:18:24.050385 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 04:18:24.050385 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 04:18:24.050385 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 04:18:24.050385 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 04:18:24.050385 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 04:18:24.050385 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 04:18:24.050385 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 04:18:24.050385 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 06 04:18:24.050385 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 06 04:18:24.050385 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 06 04:18:24.051451 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 06 04:18:24.051451 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 04:18:24.051451 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 04:18:24.051451 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 04:18:24.051451 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 04:18:24.051451 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 04:18:24.051451 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 04:18:24.051451 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 04:18:24.051451 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 04:18:24.051451 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 04:18:24.051451 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 04:18:24.051451 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 04:18:24.051451 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 04:18:24.051451 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 04:18:24.051451 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 04:18:24.051451 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 04:18:24.051451 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 04:18:24.052436 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 06 04:18:24.052436 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 06 04:18:24.052436 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 06 04:18:24.052436 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 06 04:18:24.052436 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 06 04:18:24.052436 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 06 04:18:24.052436 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 04:18:24.052436 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 04:18:24.056711 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 478/976] 10.222.0.30 () {56 vars in 1111 bytes} [Tue Sep 6 04:18:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 04:18:24.057750 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497244bd-8723-447e-b28f-304d60b204e7 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:18:24.079014 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:18:24.102755 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 499/977] 10.222.0.30 () {56 vars in 1101 bytes} [Tue Sep 6 04:18:23 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 04:18:24.173508 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497244bd-8723-447e-b28f-304d60b204e7 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:18:24.215543 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9f5d10-1240-4785-a35a-16a9c8601c10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:18:24.217005 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9f5d10-1240-4785-a35a-16a9c8601c10 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:18:24.217532 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9f5d10-1240-4785-a35a-16a9c8601c10 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:18:24.218023 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9f5d10-1240-4785-a35a-16a9c8601c10 None admin] PATH_INFO: `/v3/projects` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:18:24.218954 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9f5d10-1240-4785-a35a-16a9c8601c10 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:18:24.219494 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9f5d10-1240-4785-a35a-16a9c8601c10 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:18:24.222316 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9f5d10-1240-4785-a35a-16a9c8601c10 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:18:24.237619 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 479/978] 10.222.0.30 () {56 vars in 1148 bytes} [Tue Sep 6 04:18:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:18:24.245867 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69720ecc-5ed6-44d4-82fd-bf842ea880f3 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:18:24.248100 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6370be45-cb49-48c1-9ec4-1f674c289e36 None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:18:24.248379 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6370be45-cb49-48c1-9ec4-1f674c289e36 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:18:24.248651 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6370be45-cb49-48c1-9ec4-1f674c289e36 None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:18:24.250328 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 480/979] 10.222.0.30 () {54 vars in 927 bytes} [Tue Sep 6 04:18:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 04:18:24.255868 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede2a051-a45c-43ef-9b59-9b39c451673b None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:18:24.256151 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede2a051-a45c-43ef-9b59-9b39c451673b None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:18:24.256480 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede2a051-a45c-43ef-9b59-9b39c451673b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:18:24.274804 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-ede2a051-a45c-43ef-9b59-9b39c451673b None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:18:24.276129 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-ede2a051-a45c-43ef-9b59-9b39c451673b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:18:24.350050 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:18:24.362480 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 481/980] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Sep 6 04:18:24 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 06 04:18:24.384699 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69720ecc-5ed6-44d4-82fd-bf842ea880f3 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:18:24.422142 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37e9fb7-5874-4f50-bdea-32901b95a3f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:18:24.422763 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37e9fb7-5874-4f50-bdea-32901b95a3f6 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:18:24.423040 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37e9fb7-5874-4f50-bdea-32901b95a3f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:18:24.423309 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37e9fb7-5874-4f50-bdea-32901b95a3f6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:18:24.424599 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37e9fb7-5874-4f50-bdea-32901b95a3f6 None admin] RBAC: Adding query filter params (role.id=356631f6a0064bc695b78e3759d55d6a, scope.project.id=b707733ce65c42edbc1fd95ad0a1c06c, user.id=da5fed17101a4935bcc38b72caf9d263) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:18:24.424897 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37e9fb7-5874-4f50-bdea-32901b95a3f6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:18:24.426402 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37e9fb7-5874-4f50-bdea-32901b95a3f6 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:18:24.432729 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 500/981] 10.222.0.30 () {56 vars in 1374 bytes} [Tue Sep 6 04:18:24 2022] GET /identity/v3/role_assignments?user.id=da5fed17101a4935bcc38b72caf9d263&scope.project.id=b707733ce65c42edbc1fd95ad0a1c06c&role.id=356631f6a0064bc695b78e3759d55d6a => generated 608 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:18:24.521352 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e88f84-0e25-4684-be09-a637d7c5b314 None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:18:24.521697 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e88f84-0e25-4684-be09-a637d7c5b314 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:18:24.521935 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e88f84-0e25-4684-be09-a637d7c5b314 None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:18:24.523448 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 482/982] 10.222.0.30 () {54 vars in 927 bytes} [Tue Sep 6 04:18:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 04:18:24.528853 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a715419b-7a89-4543-9301-88d03d9ad505 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:18:24.529145 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a715419b-7a89-4543-9301-88d03d9ad505 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:18:24.529459 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a715419b-7a89-4543-9301-88d03d9ad505 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:18:24.549178 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-a715419b-7a89-4543-9301-88d03d9ad505 None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:18:24.550795 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-a715419b-7a89-4543-9301-88d03d9ad505 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:18:24.624155 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 501/983] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Sep 6 04:18:24 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 04:18:24.643065 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e88f84-0e25-4684-be09-a637d7c5b314 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:18:24.652091 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:18:24.747055 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e88f84-0e25-4684-be09-a637d7c5b314 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:18:24.782235 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6cc61e1-57c4-4c11-a0f1-7b7e8ca1d7bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:18:24.782869 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cc61e1-57c4-4c11-a0f1-7b7e8ca1d7bb None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:18:24.783279 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cc61e1-57c4-4c11-a0f1-7b7e8ca1d7bb None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:18:24.783790 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cc61e1-57c4-4c11-a0f1-7b7e8ca1d7bb None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:18:24.784403 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6cc61e1-57c4-4c11-a0f1-7b7e8ca1d7bb None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:18:24.785801 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6cc61e1-57c4-4c11-a0f1-7b7e8ca1d7bb None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:18:24.794704 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-e6cc61e1-57c4-4c11-a0f1-7b7e8ca1d7bb None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 06 04:18:24.794704 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 04:18:24.794704 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 06 04:18:24.794704 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 06 04:18:24.794704 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 06 04:18:24.794704 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 06 04:18:24.794704 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 06 04:18:24.794704 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 04:18:24.794704 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 06 04:18:24.794704 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 04:18:24.794704 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 06 04:18:24.794704 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 04:18:24.794704 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 04:18:24.794704 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 04:18:24.794704 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 04:18:24.794704 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 04:18:24.794704 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 04:18:24.795729 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 04:18:24.795729 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 04:18:24.795729 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 04:18:24.795729 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 04:18:24.795729 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 04:18:24.795729 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 04:18:24.795729 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 06 04:18:24.795729 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 06 04:18:24.795729 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 06 04:18:24.795729 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 06 04:18:24.795729 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 04:18:24.795729 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 04:18:24.795729 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 04:18:24.795729 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 04:18:24.795729 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 04:18:24.795729 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 04:18:24.796595 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 04:18:24.796595 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 04:18:24.796595 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 04:18:24.796595 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 04:18:24.796595 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 04:18:24.796595 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 04:18:24.796595 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 04:18:24.796595 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 04:18:24.796595 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 04:18:24.796595 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 04:18:24.796595 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 06 04:18:24.796595 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 04:18:24.796595 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 06 04:18:24.796595 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 04:18:24.798056 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 483/984] 10.222.0.30 () {56 vars in 1126 bytes} [Tue Sep 6 04:18:24 2022] GET /identity/v3/services/object-store => generated 93 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 04:18:24.802568 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a715419b-7a89-4543-9301-88d03d9ad505 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:18:24.881962 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:18:24.892760 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6cc61e1-57c4-4c11-a0f1-7b7e8ca1d7bb None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:18:24.905881 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a715419b-7a89-4543-9301-88d03d9ad505 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:18:24.946506 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373b01b9-a9cd-4b7c-ace3-7d34b2735925 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:18:24.947604 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373b01b9-a9cd-4b7c-ace3-7d34b2735925 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:18:24.948116 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373b01b9-a9cd-4b7c-ace3-7d34b2735925 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:18:24.948578 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373b01b9-a9cd-4b7c-ace3-7d34b2735925 None admin] PATH_INFO: `/v3/services` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:18:24.949437 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373b01b9-a9cd-4b7c-ace3-7d34b2735925 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:18:24.949934 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373b01b9-a9cd-4b7c-ace3-7d34b2735925 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:18:24.951757 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373b01b9-a9cd-4b7c-ace3-7d34b2735925 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:18:24.961478 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 502/985] 10.222.0.30 () {56 vars in 1122 bytes} [Tue Sep 6 04:18:24 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:18:24.963169 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373b01b9-a9cd-4b7c-ace3-7d34b2735925 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:18:24.966940 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:18:25.001112 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6cc61e1-57c4-4c11-a0f1-7b7e8ca1d7bb None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:18:25.037606 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b19958-e644-4b4b-b5dc-cc90a4c1ae4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:18:25.038225 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b19958-e644-4b4b-b5dc-cc90a4c1ae4b None admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:18:25.038506 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b19958-e644-4b4b-b5dc-cc90a4c1ae4b None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:18:25.038779 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b19958-e644-4b4b-b5dc-cc90a4c1ae4b None admin] PATH_INFO: `/v3/services` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:18:25.039514 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b19958-e644-4b4b-b5dc-cc90a4c1ae4b None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:18:25.040904 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b19958-e644-4b4b-b5dc-cc90a4c1ae4b None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:18:25.068585 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373b01b9-a9cd-4b7c-ace3-7d34b2735925 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:18:25.103306 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc01c9a6-30e2-4f4b-b806-12eebc0a821e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:18:25.103920 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc01c9a6-30e2-4f4b-b806-12eebc0a821e None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:18:25.104204 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc01c9a6-30e2-4f4b-b806-12eebc0a821e None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:18:25.104478 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc01c9a6-30e2-4f4b-b806-12eebc0a821e None admin] PATH_INFO: `/v3/services` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:18:25.105165 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc01c9a6-30e2-4f4b-b806-12eebc0a821e None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:18:25.105511 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc01c9a6-30e2-4f4b-b806-12eebc0a821e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:18:25.106887 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc01c9a6-30e2-4f4b-b806-12eebc0a821e None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:18:25.112451 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 503/986] 10.222.0.30 () {56 vars in 1122 bytes} [Tue Sep 6 04:18:24 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:18:25.117989 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc01c9a6-30e2-4f4b-b806-12eebc0a821e None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:18:25.121392 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 484/987] 10.222.0.30 () {60 vars in 1141 bytes} [Tue Sep 6 04:18:24 2022] POST /identity/v3/services => generated 242 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 04:18:25.131301 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:18:25.239499 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc01c9a6-30e2-4f4b-b806-12eebc0a821e None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:18:25.286004 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4339e0f-8b70-4e21-8e75-334ff9d3a122 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:18:25.286657 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4339e0f-8b70-4e21-8e75-334ff9d3a122 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:18:25.286886 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4339e0f-8b70-4e21-8e75-334ff9d3a122 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:18:25.287153 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4339e0f-8b70-4e21-8e75-334ff9d3a122 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:18:25.287857 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4339e0f-8b70-4e21-8e75-334ff9d3a122 None admin] RBAC: Adding query filter params (interface=admin, service_id=ee9be98afa174081a4bd7524fb631214, region_id=RegionOne) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:18:25.288170 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4339e0f-8b70-4e21-8e75-334ff9d3a122 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:18:25.289569 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4339e0f-8b70-4e21-8e75-334ff9d3a122 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:18:25.295023 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 504/988] 10.222.0.30 () {56 vars in 1249 bytes} [Tue Sep 6 04:18:25 2022] GET /identity/v3/endpoints?service_id=ee9be98afa174081a4bd7524fb631214&interface=admin®ion_id=RegionOne => generated 193 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 04:18:25.301017 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b19958-e644-4b4b-b5dc-cc90a4c1ae4b None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:18:25.410779 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b19958-e644-4b4b-b5dc-cc90a4c1ae4b None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:18:25.440894 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:18:25.446651 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3920826-be34-4127-8521-992408134785 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:18:25.447269 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3920826-be34-4127-8521-992408134785 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:18:25.447544 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3920826-be34-4127-8521-992408134785 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:18:25.447810 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3920826-be34-4127-8521-992408134785 None admin] PATH_INFO: `/v3/services` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:18:25.448406 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3920826-be34-4127-8521-992408134785 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:18:25.449797 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3920826-be34-4127-8521-992408134785 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:18:25.457380 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 485/989] 10.222.0.30 () {56 vars in 1087 bytes} [Tue Sep 6 04:18:25 2022] GET /identity/v3/services => generated 1727 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 04:18:26.386221 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b555b2-fc74-4d4d-8082-c8464644513f None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:18:26.386680 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b555b2-fc74-4d4d-8082-c8464644513f None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:18:26.386793 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b555b2-fc74-4d4d-8082-c8464644513f None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:18:26.388280 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 505/990] 10.222.0.30 () {54 vars in 927 bytes} [Tue Sep 6 04:18:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 04:18:26.393920 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d11335-0667-4c19-8049-e16723300a1d None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:18:26.394323 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d11335-0667-4c19-8049-e16723300a1d None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:18:26.394489 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d11335-0667-4c19-8049-e16723300a1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:18:26.413367 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-27d11335-0667-4c19-8049-e16723300a1d None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:18:26.414613 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-27d11335-0667-4c19-8049-e16723300a1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:18:26.487990 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 486/991] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Sep 6 04:18:26 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 04:18:26.753353 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b555b2-fc74-4d4d-8082-c8464644513f None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:18:26.818407 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:18:26.858282 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b555b2-fc74-4d4d-8082-c8464644513f None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:18:26.892022 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747d4b91-ce91-44f4-a430-c80666074c34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:18:26.893188 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747d4b91-ce91-44f4-a430-c80666074c34 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:18:26.893492 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747d4b91-ce91-44f4-a430-c80666074c34 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:18:26.893765 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747d4b91-ce91-44f4-a430-c80666074c34 None admin] PATH_INFO: `/v3/services/image` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:18:26.894349 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-747d4b91-ce91-44f4-a430-c80666074c34 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:18:26.895724 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-747d4b91-ce91-44f4-a430-c80666074c34 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:18:26.904468 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-747d4b91-ce91-44f4-a430-c80666074c34 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 06 04:18:26.904468 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 04:18:26.904468 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 06 04:18:26.904468 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 06 04:18:26.904468 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 06 04:18:26.904468 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 06 04:18:26.904468 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 06 04:18:26.904468 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 04:18:26.904468 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 06 04:18:26.904468 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 04:18:26.904468 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 06 04:18:26.904468 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 04:18:26.904468 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 04:18:26.904468 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 04:18:26.904468 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 04:18:26.904468 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 04:18:26.904468 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 04:18:26.905719 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 04:18:26.905719 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 04:18:26.905719 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 04:18:26.905719 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 04:18:26.905719 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 04:18:26.905719 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 04:18:26.905719 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 06 04:18:26.905719 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 06 04:18:26.905719 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 06 04:18:26.905719 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 06 04:18:26.905719 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 04:18:26.905719 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 04:18:26.905719 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 04:18:26.905719 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 04:18:26.905719 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 04:18:26.905719 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 04:18:26.906570 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 04:18:26.906570 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 04:18:26.906570 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 04:18:26.906570 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 04:18:26.906570 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 04:18:26.906570 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 04:18:26.906570 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 04:18:26.906570 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 04:18:26.906570 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 04:18:26.906570 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 04:18:26.906570 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 06 04:18:26.906570 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 04:18:26.906570 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 06 04:18:26.906570 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 04:18:26.907769 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 506/992] 10.222.0.30 () {56 vars in 1105 bytes} [Tue Sep 6 04:18:26 2022] GET /identity/v3/services/image => generated 86 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 04:18:26.912009 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d11335-0667-4c19-8049-e16723300a1d None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:18:27.012051 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d11335-0667-4c19-8049-e16723300a1d None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:18:27.036453 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:18:27.047662 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ee5ae9-ce87-41e9-8a38-542b4ae9f5d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:18:27.050026 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ee5ae9-ce87-41e9-8a38-542b4ae9f5d4 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:18:27.050026 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ee5ae9-ce87-41e9-8a38-542b4ae9f5d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:18:27.050026 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ee5ae9-ce87-41e9-8a38-542b4ae9f5d4 None admin] PATH_INFO: `/v3/services` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:18:27.050260 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ee5ae9-ce87-41e9-8a38-542b4ae9f5d4 None admin] RBAC: Adding query filter params (name=image) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:18:27.050453 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ee5ae9-ce87-41e9-8a38-542b4ae9f5d4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:18:27.051764 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ee5ae9-ce87-41e9-8a38-542b4ae9f5d4 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:18:27.061219 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 487/993] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Sep 6 04:18:26 2022] GET /identity/v3/services?name=image => generated 122 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:18:27.062774 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747d4b91-ce91-44f4-a430-c80666074c34 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:18:27.065333 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810c10a6-6319-454d-85bf-a00fa26d7d5c None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:18:27.065669 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810c10a6-6319-454d-85bf-a00fa26d7d5c None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:18:27.065943 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810c10a6-6319-454d-85bf-a00fa26d7d5c None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:18:27.067421 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 488/994] 10.222.0.30 () {54 vars in 927 bytes} [Tue Sep 6 04:18:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 06 04:18:27.073607 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff4cabe-788d-452b-9c4f-979a5dc4d791 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:18:27.073893 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff4cabe-788d-452b-9c4f-979a5dc4d791 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:18:27.074167 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff4cabe-788d-452b-9c4f-979a5dc4d791 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:18:27.092328 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-9ff4cabe-788d-452b-9c4f-979a5dc4d791 None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:18:27.093553 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-9ff4cabe-788d-452b-9c4f-979a5dc4d791 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:18:27.174637 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747d4b91-ce91-44f4-a430-c80666074c34 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:18:27.175303 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 489/995] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Sep 6 04:18:27 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 06 04:18:27.203150 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:18:27.211449 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c724eac7-7076-44f6-a571-3e210c5d81f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:18:27.215818 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c724eac7-7076-44f6-a571-3e210c5d81f1 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:18:27.215818 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c724eac7-7076-44f6-a571-3e210c5d81f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:18:27.215818 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c724eac7-7076-44f6-a571-3e210c5d81f1 None admin] PATH_INFO: `/v3/services` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:18:27.215818 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c724eac7-7076-44f6-a571-3e210c5d81f1 None admin] RBAC: Adding query filter params (type=image) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:18:27.215818 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c724eac7-7076-44f6-a571-3e210c5d81f1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:18:27.215818 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c724eac7-7076-44f6-a571-3e210c5d81f1 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:18:27.227321 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 507/996] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Sep 6 04:18:27 2022] GET /identity/v3/services?type=image => generated 122 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 04:18:27.400805 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb6d399-340d-4af1-89d6-9301ea7134f2 None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:18:27.401273 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb6d399-340d-4af1-89d6-9301ea7134f2 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:18:27.401525 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb6d399-340d-4af1-89d6-9301ea7134f2 None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:18:27.402869 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 490/997] 10.222.0.30 () {54 vars in 927 bytes} [Tue Sep 6 04:18:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 04:18:27.408168 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff7a78a-6e6d-42a6-bd58-d2e7a5d1fc32 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:18:27.408615 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff7a78a-6e6d-42a6-bd58-d2e7a5d1fc32 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:18:27.408706 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff7a78a-6e6d-42a6-bd58-d2e7a5d1fc32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:18:27.429457 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-3ff7a78a-6e6d-42a6-bd58-d2e7a5d1fc32 None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:18:27.438922 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-3ff7a78a-6e6d-42a6-bd58-d2e7a5d1fc32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:18:27.459324 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb6d399-340d-4af1-89d6-9301ea7134f2 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:18:27.504122 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 508/998] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Sep 6 04:18:27 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 04:18:27.568017 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb6d399-340d-4af1-89d6-9301ea7134f2 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:18:27.585419 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:18:27.602045 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e34e10-c564-400c-93a8-42f4b7b9c924 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:18:27.602676 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e34e10-c564-400c-93a8-42f4b7b9c924 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:18:27.602952 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e34e10-c564-400c-93a8-42f4b7b9c924 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:18:27.603220 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e34e10-c564-400c-93a8-42f4b7b9c924 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:18:27.603797 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e34e10-c564-400c-93a8-42f4b7b9c924 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:18:27.605147 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e34e10-c564-400c-93a8-42f4b7b9c924 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:18:27.614397 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-17e34e10-c564-400c-93a8-42f4b7b9c924 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 06 04:18:27.614397 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 04:18:27.614397 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 06 04:18:27.614397 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 06 04:18:27.614397 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 06 04:18:27.614397 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 06 04:18:27.614397 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 06 04:18:27.614397 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 04:18:27.614397 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 06 04:18:27.614397 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 04:18:27.614397 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 06 04:18:27.614397 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 04:18:27.614397 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 04:18:27.614397 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 04:18:27.614397 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 04:18:27.614397 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 04:18:27.614397 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 04:18:27.615411 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 04:18:27.615411 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 04:18:27.615411 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 04:18:27.615411 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 04:18:27.615411 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 04:18:27.615411 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 04:18:27.615411 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 06 04:18:27.615411 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 06 04:18:27.615411 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 06 04:18:27.615411 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 06 04:18:27.615411 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 04:18:27.615411 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 04:18:27.615411 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 04:18:27.615411 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 04:18:27.615411 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 04:18:27.615411 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 04:18:27.618414 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 04:18:27.618414 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 04:18:27.618414 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 04:18:27.618414 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 04:18:27.618414 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 04:18:27.618414 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 04:18:27.618414 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 04:18:27.618414 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 04:18:27.618414 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 04:18:27.618414 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 04:18:27.618414 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 06 04:18:27.618414 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 04:18:27.618414 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 06 04:18:27.618414 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 04:18:27.618414 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 491/999] 10.222.0.30 () {56 vars in 1111 bytes} [Tue Sep 6 04:18:27 2022] GET /identity/v3/services/compute => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 04:18:27.621767 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff7a78a-6e6d-42a6-bd58-d2e7a5d1fc32 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:18:27.724878 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff7a78a-6e6d-42a6-bd58-d2e7a5d1fc32 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:18:27.752730 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:18:27.764520 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc30d5f0-5f2b-4fa5-8678-f3a273782691 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:18:27.765141 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc30d5f0-5f2b-4fa5-8678-f3a273782691 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:18:27.765450 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc30d5f0-5f2b-4fa5-8678-f3a273782691 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:18:27.765723 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc30d5f0-5f2b-4fa5-8678-f3a273782691 None admin] PATH_INFO: `/v3/services` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:18:27.766353 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc30d5f0-5f2b-4fa5-8678-f3a273782691 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:18:27.766652 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc30d5f0-5f2b-4fa5-8678-f3a273782691 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:18:27.768089 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc30d5f0-5f2b-4fa5-8678-f3a273782691 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:18:27.777447 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 509/1000] 10.222.0.30 () {56 vars in 1112 bytes} [Tue Sep 6 04:18:27 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:18:27.777624 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e34e10-c564-400c-93a8-42f4b7b9c924 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:18:27.778493 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc30d5f0-5f2b-4fa5-8678-f3a273782691 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:18:27.885948 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e34e10-c564-400c-93a8-42f4b7b9c924 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:18:27.896021 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc30d5f0-5f2b-4fa5-8678-f3a273782691 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:18:27.922106 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7dbae18-58d7-431c-94f0-b5304ae4af3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:18:27.922942 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7dbae18-58d7-431c-94f0-b5304ae4af3d None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:18:27.923372 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7dbae18-58d7-431c-94f0-b5304ae4af3d None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:18:27.924443 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7dbae18-58d7-431c-94f0-b5304ae4af3d None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:18:27.925338 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7dbae18-58d7-431c-94f0-b5304ae4af3d None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:18:27.926873 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7dbae18-58d7-431c-94f0-b5304ae4af3d None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:18:27.929898 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0bed709-bcb6-4b66-891f-375a48977c3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:18:27.930503 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bed709-bcb6-4b66-891f-375a48977c3b None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:18:27.930786 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bed709-bcb6-4b66-891f-375a48977c3b None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:18:27.931058 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bed709-bcb6-4b66-891f-375a48977c3b None admin] PATH_INFO: `/v3/services` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:18:27.931687 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0bed709-bcb6-4b66-891f-375a48977c3b None admin] RBAC: Adding query filter params (type=compute) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:18:27.931986 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0bed709-bcb6-4b66-891f-375a48977c3b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:18:27.933347 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0bed709-bcb6-4b66-891f-375a48977c3b None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:18:27.936283 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-e7dbae18-58d7-431c-94f0-b5304ae4af3d None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 06 04:18:27.936283 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 04:18:27.936283 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 06 04:18:27.936283 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 06 04:18:27.936283 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 06 04:18:27.936283 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 06 04:18:27.936283 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 06 04:18:27.936283 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 04:18:27.936283 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 06 04:18:27.936283 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 04:18:27.936283 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 06 04:18:27.936283 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 04:18:27.936283 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 04:18:27.936283 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 04:18:27.936283 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 04:18:27.936283 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 04:18:27.936283 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 04:18:27.937631 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 04:18:27.937631 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 04:18:27.937631 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 04:18:27.937631 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 04:18:27.937631 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 04:18:27.937631 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 04:18:27.937631 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 06 04:18:27.937631 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 06 04:18:27.937631 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 06 04:18:27.937631 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 06 04:18:27.937631 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 04:18:27.937631 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 04:18:27.937631 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 04:18:27.937631 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 04:18:27.937631 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 04:18:27.937631 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 04:18:27.938499 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 04:18:27.938499 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 04:18:27.938499 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 04:18:27.938499 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 04:18:27.938499 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 04:18:27.938499 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 04:18:27.938499 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 04:18:27.938499 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 04:18:27.938499 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 04:18:27.938499 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 04:18:27.938499 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 06 04:18:27.938499 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 04:18:27.938499 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 06 04:18:27.938499 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 04:18:27.938499 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 492/1001] 10.222.0.30 () {56 vars in 1126 bytes} [Tue Sep 6 04:18:27 2022] GET /identity/v3/services/object-store => generated 93 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 04:18:27.946747 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 510/1002] 10.222.0.30 () {56 vars in 1112 bytes} [Tue Sep 6 04:18:27 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:18:27.946926 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7dbae18-58d7-431c-94f0-b5304ae4af3d None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:18:27.947461 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0bed709-bcb6-4b66-891f-375a48977c3b None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:18:27.999897 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:18:28.002547 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:18:28.056249 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0bed709-bcb6-4b66-891f-375a48977c3b None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:18:28.065821 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7dbae18-58d7-431c-94f0-b5304ae4af3d None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:18:28.093560 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ee6612-3fb1-4e1c-9c17-96491463085c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:18:28.094182 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ee6612-3fb1-4e1c-9c17-96491463085c None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:18:28.094465 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ee6612-3fb1-4e1c-9c17-96491463085c None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:18:28.094737 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ee6612-3fb1-4e1c-9c17-96491463085c None admin] PATH_INFO: `/v3/endpoints` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:18:28.095988 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ee6612-3fb1-4e1c-9c17-96491463085c None admin] RBAC: Adding query filter params (interface=public, service_id=8480ddd9e1884ee5a0465d7bf4017ae6, region_id=RegionOne) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:18:28.096298 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ee6612-3fb1-4e1c-9c17-96491463085c None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:18:28.097822 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ee6612-3fb1-4e1c-9c17-96491463085c None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:18:28.103390 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 511/1003] 10.222.0.30 () {56 vars in 1251 bytes} [Tue Sep 6 04:18:27 2022] GET /identity/v3/endpoints?service_id=8480ddd9e1884ee5a0465d7bf4017ae6&interface=public®ion_id=RegionOne => generated 194 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 04:18:28.103933 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9644fac3-eb1e-428c-924d-b59b22903c85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:18:28.104544 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9644fac3-eb1e-428c-924d-b59b22903c85 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:18:28.104822 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9644fac3-eb1e-428c-924d-b59b22903c85 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:18:28.105089 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9644fac3-eb1e-428c-924d-b59b22903c85 None admin] PATH_INFO: `/v3/services` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:18:28.105754 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9644fac3-eb1e-428c-924d-b59b22903c85 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:18:28.106053 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9644fac3-eb1e-428c-924d-b59b22903c85 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:18:28.107396 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9644fac3-eb1e-428c-924d-b59b22903c85 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:18:28.108853 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ee6612-3fb1-4e1c-9c17-96491463085c None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:18:28.113200 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 493/1004] 10.222.0.30 () {56 vars in 1122 bytes} [Tue Sep 6 04:18:27 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:18:28.119492 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9644fac3-eb1e-428c-924d-b59b22903c85 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:18:28.215044 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ee6612-3fb1-4e1c-9c17-96491463085c None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:18:28.222168 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9644fac3-eb1e-428c-924d-b59b22903c85 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:18:28.229996 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:18:28.234048 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:18:28.255180 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c7dc0e-c511-49ab-bbcc-ad19e48fd9ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:18:28.255727 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c7dc0e-c511-49ab-bbcc-ad19e48fd9ec None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:18:28.256008 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c7dc0e-c511-49ab-bbcc-ad19e48fd9ec None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:18:28.256281 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c7dc0e-c511-49ab-bbcc-ad19e48fd9ec None admin] PATH_INFO: `/v3/services` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:18:28.256890 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c7dc0e-c511-49ab-bbcc-ad19e48fd9ec None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:18:28.258318 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c7dc0e-c511-49ab-bbcc-ad19e48fd9ec None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:18:28.259207 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09547348-dc22-493e-8a82-d56a58f8967a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:18:28.259863 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09547348-dc22-493e-8a82-d56a58f8967a None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:18:28.260149 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09547348-dc22-493e-8a82-d56a58f8967a None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:18:28.260416 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09547348-dc22-493e-8a82-d56a58f8967a None admin] PATH_INFO: `/v3/services` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:18:28.261077 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09547348-dc22-493e-8a82-d56a58f8967a None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:18:28.261401 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09547348-dc22-493e-8a82-d56a58f8967a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:18:28.262744 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09547348-dc22-493e-8a82-d56a58f8967a None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:18:28.266834 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 512/1005] 10.222.0.30 () {56 vars in 1087 bytes} [Tue Sep 6 04:18:28 2022] GET /identity/v3/services => generated 1727 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 06 04:18:28.272673 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 494/1006] 10.222.0.30 () {56 vars in 1122 bytes} [Tue Sep 6 04:18:28 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 04:18:28.273630 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c7dc0e-c511-49ab-bbcc-ad19e48fd9ec None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:18:28.396380 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c7dc0e-c511-49ab-bbcc-ad19e48fd9ec None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:18:28.434992 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1d7260-e667-4f36-b539-2683cec7f8e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:18:28.435634 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1d7260-e667-4f36-b539-2683cec7f8e0 None admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:18:28.435919 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1d7260-e667-4f36-b539-2683cec7f8e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:18:28.436194 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1d7260-e667-4f36-b539-2683cec7f8e0 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:18:28.436954 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa1d7260-e667-4f36-b539-2683cec7f8e0 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:18:28.438413 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa1d7260-e667-4f36-b539-2683cec7f8e0 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:18:28.530323 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 513/1007] 10.222.0.30 () {60 vars in 1144 bytes} [Tue Sep 6 04:18:28 2022] POST /identity/v3/endpoints => generated 325 bytes in 257 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 04:18:29.142074 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d49e4d6-515f-4851-ba80-f97d3b5dcd2e None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:18:29.143579 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d49e4d6-515f-4851-ba80-f97d3b5dcd2e None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:18:29.143579 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d49e4d6-515f-4851-ba80-f97d3b5dcd2e None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:18:29.144637 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 495/1008] 10.222.0.30 () {54 vars in 927 bytes} [Tue Sep 6 04:18:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 04:18:29.149629 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1ae769-3f30-433a-b587-376a0e3c915c None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:18:29.149985 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1ae769-3f30-433a-b587-376a0e3c915c None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:18:29.150336 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1ae769-3f30-433a-b587-376a0e3c915c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:18:29.169482 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-6c1ae769-3f30-433a-b587-376a0e3c915c None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:18:29.170674 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-6c1ae769-3f30-433a-b587-376a0e3c915c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:18:29.201652 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:18:29.253817 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 514/1009] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Sep 6 04:18:29 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 04:18:29.527734 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d49e4d6-515f-4851-ba80-f97d3b5dcd2e None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:18:29.634756 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d49e4d6-515f-4851-ba80-f97d3b5dcd2e None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:18:29.669613 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d0485e-cee8-4565-841f-cd25569e729b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:18:29.670225 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d0485e-cee8-4565-841f-cd25569e729b None admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:18:29.670506 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d0485e-cee8-4565-841f-cd25569e729b None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:18:29.670777 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d0485e-cee8-4565-841f-cd25569e729b None admin] PATH_INFO: `/v3/services` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:18:29.671440 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d0485e-cee8-4565-841f-cd25569e729b None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:18:29.672806 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d0485e-cee8-4565-841f-cd25569e729b None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:18:29.767253 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 496/1010] 10.222.0.30 () {60 vars in 1141 bytes} [Tue Sep 6 04:18:29 2022] POST /identity/v3/services => generated 242 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 04:18:30.226851 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce56e0ea-f2a9-493f-81ff-29e062f6c7a0 None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:18:30.227356 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce56e0ea-f2a9-493f-81ff-29e062f6c7a0 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:18:30.227434 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce56e0ea-f2a9-493f-81ff-29e062f6c7a0 None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:18:30.228945 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 515/1011] 10.222.0.30 () {54 vars in 927 bytes} [Tue Sep 6 04:18:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 04:18:30.234070 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746cf4bf-6233-48f2-90c8-866645dad869 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:18:30.234511 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746cf4bf-6233-48f2-90c8-866645dad869 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:18:30.234592 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746cf4bf-6233-48f2-90c8-866645dad869 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:18:30.253335 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-746cf4bf-6233-48f2-90c8-866645dad869 None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:18:30.254535 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-746cf4bf-6233-48f2-90c8-866645dad869 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:18:30.281438 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:18:30.332917 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 497/1012] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Sep 6 04:18:30 2022] POST /identity/v3/auth/tokens => generated 1813 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 06 04:18:30.474712 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324e5369-6b3e-426c-add6-d8259303557e None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:18:30.475028 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324e5369-6b3e-426c-add6-d8259303557e None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:18:30.475275 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324e5369-6b3e-426c-add6-d8259303557e None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:18:30.476763 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 516/1013] 10.222.0.30 () {54 vars in 927 bytes} [Tue Sep 6 04:18:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 06 04:18:30.481963 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3722fe-f16c-43fe-a3ea-1630525cb31b None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:18:30.482209 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3722fe-f16c-43fe-a3ea-1630525cb31b None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:18:30.482487 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3722fe-f16c-43fe-a3ea-1630525cb31b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:18:30.500650 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-3f3722fe-f16c-43fe-a3ea-1630525cb31b None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:18:30.502613 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-3f3722fe-f16c-43fe-a3ea-1630525cb31b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:18:30.569819 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 498/1014] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Sep 6 04:18:30 2022] POST /identity/v3/auth/tokens => generated 1813 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 04:18:30.598002 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324e5369-6b3e-426c-add6-d8259303557e None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:18:30.698405 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324e5369-6b3e-426c-add6-d8259303557e None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:18:30.732072 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2ebe71-04e6-4357-97dd-51cd986eebee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:18:30.732631 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2ebe71-04e6-4357-97dd-51cd986eebee None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:18:30.732908 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2ebe71-04e6-4357-97dd-51cd986eebee None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:18:30.733174 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2ebe71-04e6-4357-97dd-51cd986eebee None admin] PATH_INFO: `/v3/services/compute` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:18:30.733779 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2ebe71-04e6-4357-97dd-51cd986eebee None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:18:30.735138 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2ebe71-04e6-4357-97dd-51cd986eebee None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:18:30.747722 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-1c2ebe71-04e6-4357-97dd-51cd986eebee None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 06 04:18:30.747722 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 04:18:30.747722 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 06 04:18:30.747722 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 06 04:18:30.747722 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 06 04:18:30.747722 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 06 04:18:30.747722 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 06 04:18:30.747722 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 04:18:30.747722 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 06 04:18:30.747722 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 04:18:30.747722 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 06 04:18:30.747722 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 04:18:30.747722 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 04:18:30.747722 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 04:18:30.747722 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 04:18:30.747722 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 04:18:30.747722 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 04:18:30.748761 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 04:18:30.748761 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 04:18:30.748761 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 04:18:30.748761 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 04:18:30.748761 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 04:18:30.748761 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 04:18:30.748761 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 06 04:18:30.748761 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 06 04:18:30.748761 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 06 04:18:30.748761 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 06 04:18:30.748761 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 04:18:30.748761 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 04:18:30.748761 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 04:18:30.748761 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 04:18:30.748761 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 04:18:30.748761 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 04:18:30.749726 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 04:18:30.749726 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 04:18:30.749726 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 04:18:30.749726 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 04:18:30.749726 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 04:18:30.749726 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 04:18:30.749726 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 04:18:30.749726 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 04:18:30.749726 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 04:18:30.749726 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 04:18:30.749726 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 06 04:18:30.749726 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 04:18:30.749726 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 06 04:18:30.749726 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 04:18:30.753668 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 517/1015] 10.222.0.30 () {56 vars in 1111 bytes} [Tue Sep 6 04:18:30 2022] GET /identity/v3/services/compute => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 04:18:30.755333 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3722fe-f16c-43fe-a3ea-1630525cb31b None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:18:30.840908 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2ebe71-04e6-4357-97dd-51cd986eebee None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:18:30.847151 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:18:30.857240 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3722fe-f16c-43fe-a3ea-1630525cb31b None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:18:30.873683 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:18:30.895185 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983ed581-e5e9-4a96-bdb1-863599a6c2bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:18:30.896005 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983ed581-e5e9-4a96-bdb1-863599a6c2bb None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:18:30.897365 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983ed581-e5e9-4a96-bdb1-863599a6c2bb None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:18:30.898037 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983ed581-e5e9-4a96-bdb1-863599a6c2bb None admin] PATH_INFO: `/v3/services` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:18:30.898704 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983ed581-e5e9-4a96-bdb1-863599a6c2bb None admin] RBAC: Adding query filter params (name=compute) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:18:30.899014 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983ed581-e5e9-4a96-bdb1-863599a6c2bb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:18:30.900381 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983ed581-e5e9-4a96-bdb1-863599a6c2bb None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:18:30.905536 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 499/1016] 10.222.0.30 () {56 vars in 1112 bytes} [Tue Sep 6 04:18:30 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:18:30.910949 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983ed581-e5e9-4a96-bdb1-863599a6c2bb None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:18:30.949575 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2ebe71-04e6-4357-97dd-51cd986eebee None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:18:30.984896 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78dbca1-01d9-466f-9181-7648abb81da9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:18:30.985931 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78dbca1-01d9-466f-9181-7648abb81da9 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:18:30.986739 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78dbca1-01d9-466f-9181-7648abb81da9 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:18:30.987051 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78dbca1-01d9-466f-9181-7648abb81da9 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:18:30.988524 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f78dbca1-01d9-466f-9181-7648abb81da9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:18:30.990316 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f78dbca1-01d9-466f-9181-7648abb81da9 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:18:30.992030 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 518/1017] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Sep 6 04:18:30 2022] GET /identity/v3/domains/default => generated 199 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 04:18:30.997458 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78dbca1-01d9-466f-9181-7648abb81da9 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:18:31.010579 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983ed581-e5e9-4a96-bdb1-863599a6c2bb None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:18:31.045574 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4553bb-bd56-4154-8910-e60ed24ca2a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:18:31.046187 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4553bb-bd56-4154-8910-e60ed24ca2a5 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:18:31.046465 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4553bb-bd56-4154-8910-e60ed24ca2a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:18:31.046732 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4553bb-bd56-4154-8910-e60ed24ca2a5 None admin] PATH_INFO: `/v3/services` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:18:31.047354 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4553bb-bd56-4154-8910-e60ed24ca2a5 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:18:31.047642 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4553bb-bd56-4154-8910-e60ed24ca2a5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:18:31.049730 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4553bb-bd56-4154-8910-e60ed24ca2a5 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:18:31.054584 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 500/1018] 10.222.0.30 () {56 vars in 1112 bytes} [Tue Sep 6 04:18:30 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 04:18:31.060073 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4553bb-bd56-4154-8910-e60ed24ca2a5 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:18:31.079546 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:18:31.106261 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:18:31.113137 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78dbca1-01d9-466f-9181-7648abb81da9 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:18:31.150019 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f61dc3-c100-423d-9761-620992448f31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:18:31.150626 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f61dc3-c100-423d-9761-620992448f31 None admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:18:31.150905 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f61dc3-c100-423d-9761-620992448f31 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:18:31.151168 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f61dc3-c100-423d-9761-620992448f31 None admin] PATH_INFO: `/v3/projects` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:18:31.151815 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f61dc3-c100-423d-9761-620992448f31 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:18:31.153235 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f61dc3-c100-423d-9761-620992448f31 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:18:31.171561 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4553bb-bd56-4154-8910-e60ed24ca2a5 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:18:31.208222 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3d8b90-058e-4cb6-8fe1-e8353e9ead8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:18:31.208825 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3d8b90-058e-4cb6-8fe1-e8353e9ead8b None admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:18:31.209138 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3d8b90-058e-4cb6-8fe1-e8353e9ead8b None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:18:31.209404 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3d8b90-058e-4cb6-8fe1-e8353e9ead8b None admin] PATH_INFO: `/v3/endpoints` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:18:31.210083 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3d8b90-058e-4cb6-8fe1-e8353e9ead8b None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:18:31.211551 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3d8b90-058e-4cb6-8fe1-e8353e9ead8b None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:18:31.239181 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 519/1019] 10.222.0.30 () {60 vars in 1141 bytes} [Tue Sep 6 04:18:30 2022] POST /identity/v3/projects => generated 311 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 04:18:31.287955 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 501/1020] 10.222.0.30 () {60 vars in 1144 bytes} [Tue Sep 6 04:18:31 2022] POST /identity/v3/endpoints => generated 334 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 04:18:31.710952 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e087b4-b405-4a21-9397-825cebcbb250 None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:18:31.711261 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e087b4-b405-4a21-9397-825cebcbb250 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:18:31.711607 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e087b4-b405-4a21-9397-825cebcbb250 None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:18:31.713096 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 520/1021] 10.222.0.30 () {54 vars in 927 bytes} [Tue Sep 6 04:18:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 04:18:31.718071 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8d8086-4ee1-4525-a4ff-00ed44060bed None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:18:31.718358 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8d8086-4ee1-4525-a4ff-00ed44060bed None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:18:31.718626 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8d8086-4ee1-4525-a4ff-00ed44060bed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:18:31.737088 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-6f8d8086-4ee1-4525-a4ff-00ed44060bed None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:18:31.738298 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-6f8d8086-4ee1-4525-a4ff-00ed44060bed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:18:31.810410 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:18:31.818287 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 502/1022] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Sep 6 04:18:31 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 04:18:32.091430 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e087b4-b405-4a21-9397-825cebcbb250 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:18:32.198828 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e087b4-b405-4a21-9397-825cebcbb250 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:18:32.233136 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76df4bea-d971-48fd-be78-e91ad48b1982 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:18:32.233786 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76df4bea-d971-48fd-be78-e91ad48b1982 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:18:32.234067 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76df4bea-d971-48fd-be78-e91ad48b1982 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:18:32.234339 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76df4bea-d971-48fd-be78-e91ad48b1982 None admin] PATH_INFO: `/v3/services/image` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:18:32.234919 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76df4bea-d971-48fd-be78-e91ad48b1982 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:18:32.236771 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76df4bea-d971-48fd-be78-e91ad48b1982 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:18:32.245851 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-76df4bea-d971-48fd-be78-e91ad48b1982 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 06 04:18:32.245851 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 04:18:32.245851 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 06 04:18:32.245851 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 06 04:18:32.245851 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 06 04:18:32.245851 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 06 04:18:32.245851 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 06 04:18:32.245851 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 04:18:32.245851 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 06 04:18:32.245851 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 04:18:32.245851 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 06 04:18:32.245851 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 04:18:32.245851 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 04:18:32.245851 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 04:18:32.245851 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 04:18:32.245851 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 04:18:32.245851 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 04:18:32.247365 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 04:18:32.247365 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 04:18:32.247365 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 04:18:32.247365 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 04:18:32.247365 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 04:18:32.247365 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 04:18:32.247365 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 06 04:18:32.247365 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 06 04:18:32.247365 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 06 04:18:32.247365 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 06 04:18:32.247365 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 04:18:32.247365 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 04:18:32.247365 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 04:18:32.247365 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 04:18:32.247365 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 04:18:32.247365 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 04:18:32.248198 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 04:18:32.248198 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 04:18:32.248198 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 04:18:32.248198 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 04:18:32.248198 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 04:18:32.248198 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 04:18:32.248198 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 04:18:32.248198 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 04:18:32.248198 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 04:18:32.248198 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 04:18:32.248198 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 06 04:18:32.248198 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 04:18:32.248198 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 06 04:18:32.248198 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 04:18:32.249629 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 521/1023] 10.222.0.30 () {56 vars in 1105 bytes} [Tue Sep 6 04:18:32 2022] GET /identity/v3/services/image => generated 86 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 04:18:32.253153 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8d8086-4ee1-4525-a4ff-00ed44060bed None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:18:32.353798 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8d8086-4ee1-4525-a4ff-00ed44060bed None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:18:32.396769 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5f3fa4-2c9e-4024-a5d2-f8a2c0408365 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:18:32.397449 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5f3fa4-2c9e-4024-a5d2-f8a2c0408365 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:18:32.397734 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5f3fa4-2c9e-4024-a5d2-f8a2c0408365 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:18:32.398009 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5f3fa4-2c9e-4024-a5d2-f8a2c0408365 None admin] PATH_INFO: `/v3/services` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:18:32.398657 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf5f3fa4-2c9e-4024-a5d2-f8a2c0408365 None admin] RBAC: Adding query filter params (name=image) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:18:32.398961 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf5f3fa4-2c9e-4024-a5d2-f8a2c0408365 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:18:32.400342 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf5f3fa4-2c9e-4024-a5d2-f8a2c0408365 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:18:32.410995 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 503/1024] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Sep 6 04:18:32 2022] GET /identity/v3/services?name=image => generated 122 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:18:32.412196 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76df4bea-d971-48fd-be78-e91ad48b1982 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:18:32.427958 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:18:32.515481 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76df4bea-d971-48fd-be78-e91ad48b1982 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:18:32.550804 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efbe868-1a86-4300-b882-b82183b867c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:18:32.551412 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efbe868-1a86-4300-b882-b82183b867c4 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:18:32.551693 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efbe868-1a86-4300-b882-b82183b867c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:18:32.551966 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efbe868-1a86-4300-b882-b82183b867c4 None admin] PATH_INFO: `/v3/services` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:18:32.552601 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0efbe868-1a86-4300-b882-b82183b867c4 None admin] RBAC: Adding query filter params (type=image) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:18:32.552906 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0efbe868-1a86-4300-b882-b82183b867c4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:18:32.554315 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0efbe868-1a86-4300-b882-b82183b867c4 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:18:32.563883 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 522/1025] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Sep 6 04:18:32 2022] GET /identity/v3/services?type=image => generated 350 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:18:32.565348 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5f3fa4-2c9e-4024-a5d2-f8a2c0408365 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:18:32.627227 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:18:32.666793 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5f3fa4-2c9e-4024-a5d2-f8a2c0408365 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:18:32.700457 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c42896-ee11-4aaf-968d-c262e8a432b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:18:32.701065 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c42896-ee11-4aaf-968d-c262e8a432b4 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:18:32.701377 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c42896-ee11-4aaf-968d-c262e8a432b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:18:32.701654 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c42896-ee11-4aaf-968d-c262e8a432b4 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:18:32.702351 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c42896-ee11-4aaf-968d-c262e8a432b4 None admin] RBAC: Adding query filter params (interface=public, service_id=f99a89a6d980463b82b8912e3ca20e94, region_id=RegionOne) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:18:32.702656 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c42896-ee11-4aaf-968d-c262e8a432b4 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:18:32.704018 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c42896-ee11-4aaf-968d-c262e8a432b4 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:18:32.709826 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 504/1026] 10.222.0.30 () {56 vars in 1251 bytes} [Tue Sep 6 04:18:32 2022] GET /identity/v3/endpoints?service_id=f99a89a6d980463b82b8912e3ca20e94&interface=public®ion_id=RegionOne => generated 194 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 04:18:32.715234 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efbe868-1a86-4300-b882-b82183b867c4 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:18:32.798010 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:18:32.822241 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efbe868-1a86-4300-b882-b82183b867c4 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:18:32.857428 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5a4b8a-2698-48cd-8f27-9571df66cfcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:18:32.858396 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5a4b8a-2698-48cd-8f27-9571df66cfcf None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:18:32.858904 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5a4b8a-2698-48cd-8f27-9571df66cfcf None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:18:32.859374 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5a4b8a-2698-48cd-8f27-9571df66cfcf None admin] PATH_INFO: `/v3/services` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:18:32.860208 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e5a4b8a-2698-48cd-8f27-9571df66cfcf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:18:32.862467 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e5a4b8a-2698-48cd-8f27-9571df66cfcf None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:18:32.877423 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 523/1027] 10.222.0.30 () {56 vars in 1087 bytes} [Tue Sep 6 04:18:32 2022] GET /identity/v3/services => generated 1957 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 04:18:33.235560 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ece105-9141-473b-b57b-38418305fe89 None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:18:33.235753 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ece105-9141-473b-b57b-38418305fe89 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:18:33.236501 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ece105-9141-473b-b57b-38418305fe89 None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:18:33.237721 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 505/1028] 10.222.0.30 () {54 vars in 927 bytes} [Tue Sep 6 04:18:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 04:18:33.242816 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817854f3-1567-46f7-b7e6-1178e0e82eb0 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:18:33.243102 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817854f3-1567-46f7-b7e6-1178e0e82eb0 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:18:33.243375 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817854f3-1567-46f7-b7e6-1178e0e82eb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:18:33.269105 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-817854f3-1567-46f7-b7e6-1178e0e82eb0 None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:18:33.270331 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-817854f3-1567-46f7-b7e6-1178e0e82eb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:18:33.309370 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc810117-62ca-4595-81fc-8b84f380ae35 None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:18:33.309671 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc810117-62ca-4595-81fc-8b84f380ae35 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:18:33.309924 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc810117-62ca-4595-81fc-8b84f380ae35 None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:18:33.311396 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 506/1029] 10.222.0.30 () {54 vars in 927 bytes} [Tue Sep 6 04:18:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 06 04:18:33.316710 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeeef009-ffac-4b65-b2a3-14624acfd92d None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:18:33.317002 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeeef009-ffac-4b65-b2a3-14624acfd92d None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:18:33.317276 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeeef009-ffac-4b65-b2a3-14624acfd92d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:18:33.341981 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-eeeef009-ffac-4b65-b2a3-14624acfd92d None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:18:33.342210 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 524/1030] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Sep 6 04:18:33 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 06 04:18:33.343860 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-eeeef009-ffac-4b65-b2a3-14624acfd92d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:18:33.421858 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 507/1031] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Sep 6 04:18:33 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 04:18:33.608606 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817854f3-1567-46f7-b7e6-1178e0e82eb0 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:18:33.680287 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:18:33.687611 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeeef009-ffac-4b65-b2a3-14624acfd92d None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:18:33.711834 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817854f3-1567-46f7-b7e6-1178e0e82eb0 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:18:33.748196 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20787bf6-e4f7-4994-9c47-532c29b2f19d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:18:33.748828 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20787bf6-e4f7-4994-9c47-532c29b2f19d None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:18:33.749107 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20787bf6-e4f7-4994-9c47-532c29b2f19d None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:18:33.749407 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20787bf6-e4f7-4994-9c47-532c29b2f19d None admin] PATH_INFO: `/v3/domains/default` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:18:33.750508 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20787bf6-e4f7-4994-9c47-532c29b2f19d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:18:33.752038 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:18:33.752558 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20787bf6-e4f7-4994-9c47-532c29b2f19d None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:18:33.758819 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 525/1032] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Sep 6 04:18:33 2022] GET /identity/v3/domains/default => generated 199 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:18:33.759806 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20787bf6-e4f7-4994-9c47-532c29b2f19d None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:18:33.794552 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeeef009-ffac-4b65-b2a3-14624acfd92d None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:18:33.828540 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702cda03-06e4-4005-b837-47eeb8ed0a05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:18:33.829183 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702cda03-06e4-4005-b837-47eeb8ed0a05 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:18:33.830029 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702cda03-06e4-4005-b837-47eeb8ed0a05 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:18:33.830305 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702cda03-06e4-4005-b837-47eeb8ed0a05 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:18:33.840801 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-702cda03-06e4-4005-b837-47eeb8ed0a05 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 04:18:33.845840 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-702cda03-06e4-4005-b837-47eeb8ed0a05 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 06 04:18:33.845840 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 04:18:33.845840 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 04:18:33.845840 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 04:18:33.845840 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 04:18:33.845840 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 04:18:33.845840 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 04:18:33.845840 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 04:18:33.845840 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 04:18:33.845840 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 04:18:33.845840 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 04:18:33.845840 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 04:18:33.845840 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 04:18:33.845840 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 04:18:33.845840 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 04:18:33.845840 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 04:18:33.849467 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 04:18:33.849467 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 04:18:33.849467 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 04:18:33.849467 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 04:18:33.849467 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 04:18:33.849467 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 04:18:33.849467 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 04:18:33.849467 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 04:18:33.849467 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 04:18:33.849467 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 04:18:33.849467 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 04:18:33.849467 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 04:18:33.849467 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 04:18:33.849467 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 04:18:33.849467 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 04:18:33.849467 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 04:18:33.850339 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 04:18:33.850339 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 04:18:33.850339 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 04:18:33.850339 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 04:18:33.850339 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 04:18:33.850339 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 04:18:33.850339 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 04:18:33.850339 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 04:18:33.850339 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 04:18:33.850339 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 04:18:33.850339 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 06 04:18:33.850339 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 04:18:33.850339 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 508/1033] 10.222.0.30 () {56 vars in 1120 bytes} [Tue Sep 6 04:18:33 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 04:18:33.852778 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702cda03-06e4-4005-b837-47eeb8ed0a05 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:18:33.865756 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20787bf6-e4f7-4994-9c47-532c29b2f19d None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:18:33.903096 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7babaa7b-d1bc-4a7d-987e-4e76ff51b1b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:18:33.903713 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7babaa7b-d1bc-4a7d-987e-4e76ff51b1b1 None admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:18:33.904170 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7babaa7b-d1bc-4a7d-987e-4e76ff51b1b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:18:33.904451 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7babaa7b-d1bc-4a7d-987e-4e76ff51b1b1 None admin] PATH_INFO: `/v3/users` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:18:33.905230 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7babaa7b-d1bc-4a7d-987e-4e76ff51b1b1 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:18:33.906712 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7babaa7b-d1bc-4a7d-987e-4e76ff51b1b1 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:18:33.918033 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:18:33.956295 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702cda03-06e4-4005-b837-47eeb8ed0a05 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:18:33.975445 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:18:33.996717 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de94639-71cf-4e32-b4d4-a47cb3bce35f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:18:33.997354 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de94639-71cf-4e32-b4d4-a47cb3bce35f None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:18:33.997637 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de94639-71cf-4e32-b4d4-a47cb3bce35f None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:18:33.997906 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de94639-71cf-4e32-b4d4-a47cb3bce35f None admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:18:33.998572 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de94639-71cf-4e32-b4d4-a47cb3bce35f None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:18:33.999007 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de94639-71cf-4e32-b4d4-a47cb3bce35f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:18:34.000533 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de94639-71cf-4e32-b4d4-a47cb3bce35f None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:18:34.009159 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 509/1034] 10.222.0.30 () {56 vars in 1115 bytes} [Tue Sep 6 04:18:33 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 04:18:34.015072 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de94639-71cf-4e32-b4d4-a47cb3bce35f None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:18:34.049331 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 526/1035] 10.222.0.30 () {60 vars in 1132 bytes} [Tue Sep 6 04:18:33 2022] POST /identity/v3/users => generated 285 bytes in 281 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 04:18:34.123401 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de94639-71cf-4e32-b4d4-a47cb3bce35f None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:18:34.159055 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dca44aa-2a53-4b01-888f-64e7177c729f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:18:34.159697 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dca44aa-2a53-4b01-888f-64e7177c729f None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:18:34.159974 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dca44aa-2a53-4b01-888f-64e7177c729f None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:18:34.160243 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dca44aa-2a53-4b01-888f-64e7177c729f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:18:34.171706 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dca44aa-2a53-4b01-888f-64e7177c729f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:18:34.173162 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dca44aa-2a53-4b01-888f-64e7177c729f None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:18:34.188766 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-0dca44aa-2a53-4b01-888f-64e7177c729f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 04:18:34.188766 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 04:18:34.188766 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 04:18:34.188766 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 04:18:34.188766 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 04:18:34.188766 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 04:18:34.188766 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 04:18:34.188766 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 04:18:34.188766 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 04:18:34.188766 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 04:18:34.188766 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 04:18:34.188766 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 04:18:34.188766 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 04:18:34.188766 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 04:18:34.188766 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 04:18:34.188766 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 04:18:34.190145 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 04:18:34.190145 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 04:18:34.190145 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 04:18:34.190145 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 04:18:34.190145 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 04:18:34.190145 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 04:18:34.190145 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 04:18:34.190145 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 04:18:34.190145 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 04:18:34.190145 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 04:18:34.190145 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 04:18:34.190145 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 04:18:34.190145 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 04:18:34.190145 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 04:18:34.190145 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 04:18:34.190145 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 04:18:34.190145 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 04:18:34.191063 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 04:18:34.191063 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 04:18:34.191063 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 04:18:34.191063 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 510/1036] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Sep 6 04:18:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 04:18:34.195986 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7babaa7b-d1bc-4a7d-987e-4e76ff51b1b1 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:18:34.302808 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7babaa7b-d1bc-4a7d-987e-4e76ff51b1b1 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:18:34.337741 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0053087f-bd7b-4304-b12e-5f2b301d909b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:18:34.338349 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0053087f-bd7b-4304-b12e-5f2b301d909b None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:18:34.338624 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0053087f-bd7b-4304-b12e-5f2b301d909b None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:18:34.338892 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0053087f-bd7b-4304-b12e-5f2b301d909b None admin] PATH_INFO: `/v3/domains` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:18:34.339534 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0053087f-bd7b-4304-b12e-5f2b301d909b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:18:34.339832 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0053087f-bd7b-4304-b12e-5f2b301d909b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:18:34.341190 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0053087f-bd7b-4304-b12e-5f2b301d909b None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:18:34.354079 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 527/1037] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Sep 6 04:18:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:18:34.360062 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dca44aa-2a53-4b01-888f-64e7177c729f None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:18:34.403929 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:18:34.476931 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dca44aa-2a53-4b01-888f-64e7177c729f None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:18:34.511489 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ddfe60-f601-470b-8a1a-4b77e6cd4f22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:18:34.512129 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ddfe60-f601-470b-8a1a-4b77e6cd4f22 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:18:34.512410 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ddfe60-f601-470b-8a1a-4b77e6cd4f22 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:18:34.512684 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ddfe60-f601-470b-8a1a-4b77e6cd4f22 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:18:34.520884 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ddfe60-f601-470b-8a1a-4b77e6cd4f22 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:18:34.522373 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ddfe60-f601-470b-8a1a-4b77e6cd4f22 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:18:34.533873 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-45ddfe60-f601-470b-8a1a-4b77e6cd4f22 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 06 04:18:34.533873 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 04:18:34.533873 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 04:18:34.533873 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 04:18:34.533873 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 04:18:34.533873 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 04:18:34.533873 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 04:18:34.533873 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 04:18:34.533873 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 04:18:34.533873 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 04:18:34.533873 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 04:18:34.533873 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 04:18:34.533873 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 06 04:18:34.533873 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 06 04:18:34.533873 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 06 04:18:34.533873 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 06 04:18:34.534858 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 04:18:34.534858 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 04:18:34.534858 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 06 04:18:34.534858 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 04:18:34.534858 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 06 04:18:34.534858 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 04:18:34.534858 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 04:18:34.534858 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 04:18:34.534858 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 04:18:34.534858 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 04:18:34.534858 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 04:18:34.534858 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 04:18:34.534858 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 04:18:34.534858 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 04:18:34.534858 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 04:18:34.534858 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 04:18:34.534858 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 04:18:34.534858 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 04:18:34.535902 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 04:18:34.535902 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 04:18:34.535902 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 06 04:18:34.535902 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 06 04:18:34.535902 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 06 04:18:34.535902 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 06 04:18:34.535902 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 06 04:18:34.535902 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 06 04:18:34.535902 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 06 04:18:34.535902 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 04:18:34.537391 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 511/1038] 10.222.0.30 () {56 vars in 1093 bytes} [Tue Sep 6 04:18:34 2022] GET /identity/v3/users/nova => generated 82 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 04:18:34.541109 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0053087f-bd7b-4304-b12e-5f2b301d909b None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:18:34.599578 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:18:34.644816 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0053087f-bd7b-4304-b12e-5f2b301d909b None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:18:34.680234 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838e89f9-dfc9-4c84-bdac-d7476a2f98dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:18:34.680856 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838e89f9-dfc9-4c84-bdac-d7476a2f98dc None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:18:34.681137 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838e89f9-dfc9-4c84-bdac-d7476a2f98dc None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:18:34.681440 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838e89f9-dfc9-4c84-bdac-d7476a2f98dc None admin] PATH_INFO: `/v3/users` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:18:34.682181 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838e89f9-dfc9-4c84-bdac-d7476a2f98dc None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:18:34.682488 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838e89f9-dfc9-4c84-bdac-d7476a2f98dc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:18:34.684001 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838e89f9-dfc9-4c84-bdac-d7476a2f98dc None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:18:34.714592 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 528/1039] 10.222.0.30 () {56 vars in 1133 bytes} [Tue Sep 6 04:18:34 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:18:34.716214 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ddfe60-f601-470b-8a1a-4b77e6cd4f22 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:18:34.812543 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:18:34.813628 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6306fee-70dc-44e9-b502-5d5441b3e389 None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:18:34.814453 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6306fee-70dc-44e9-b502-5d5441b3e389 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:18:34.814857 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6306fee-70dc-44e9-b502-5d5441b3e389 None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:18:34.816672 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 529/1040] 10.222.0.30 () {54 vars in 927 bytes} [Tue Sep 6 04:18:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 06 04:18:34.821478 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d2547d-731c-45ac-9149-f07eb8dc117a None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:18:34.824273 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d2547d-731c-45ac-9149-f07eb8dc117a None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:18:34.824273 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d2547d-731c-45ac-9149-f07eb8dc117a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:18:34.824432 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ddfe60-f601-470b-8a1a-4b77e6cd4f22 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:18:34.840742 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-47d2547d-731c-45ac-9149-f07eb8dc117a None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:18:34.842015 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-47d2547d-731c-45ac-9149-f07eb8dc117a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:18:34.859619 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc9a94c-bdb9-4417-a604-0399392f4cc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:18:34.860232 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc9a94c-bdb9-4417-a604-0399392f4cc7 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:18:34.860509 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc9a94c-bdb9-4417-a604-0399392f4cc7 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:18:34.860845 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc9a94c-bdb9-4417-a604-0399392f4cc7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:18:34.921547 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 530/1041] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Sep 6 04:18:34 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 06 04:18:34.928180 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc9a94c-bdb9-4417-a604-0399392f4cc7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:18:34.929747 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc9a94c-bdb9-4417-a604-0399392f4cc7 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:18:35.019125 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:18:35.028985 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 512/1042] 10.222.0.30 () {58 vars in 1282 bytes} [Tue Sep 6 04:18:34 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 06 04:18:35.034749 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d2547d-731c-45ac-9149-f07eb8dc117a None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:18:35.079470 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:18:35.149734 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d2547d-731c-45ac-9149-f07eb8dc117a None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:18:35.184764 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c122b14-8c88-4363-b0d5-3784e037c9ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:18:35.185337 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c122b14-8c88-4363-b0d5-3784e037c9ca None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:18:35.185615 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c122b14-8c88-4363-b0d5-3784e037c9ca None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:18:35.185884 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c122b14-8c88-4363-b0d5-3784e037c9ca None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:18:35.188012 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc9a94c-bdb9-4417-a604-0399392f4cc7 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:18:35.197555 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c122b14-8c88-4363-b0d5-3784e037c9ca None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:18:35.199155 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c122b14-8c88-4363-b0d5-3784e037c9ca None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:18:35.214857 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-9c122b14-8c88-4363-b0d5-3784e037c9ca None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 04:18:35.214857 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 04:18:35.214857 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 04:18:35.214857 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 04:18:35.214857 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 04:18:35.214857 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 04:18:35.214857 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 04:18:35.214857 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 04:18:35.214857 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 04:18:35.214857 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 04:18:35.214857 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 04:18:35.214857 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 04:18:35.214857 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 04:18:35.214857 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 04:18:35.214857 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 04:18:35.214857 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 04:18:35.215845 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 04:18:35.215845 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 04:18:35.215845 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 04:18:35.215845 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 04:18:35.215845 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 04:18:35.215845 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 04:18:35.215845 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 04:18:35.215845 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 04:18:35.215845 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 04:18:35.215845 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 04:18:35.215845 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 04:18:35.215845 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 04:18:35.215845 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 04:18:35.215845 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 04:18:35.215845 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 04:18:35.215845 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 04:18:35.215845 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 04:18:35.216774 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 04:18:35.216774 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 04:18:35.216774 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 04:18:35.216774 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 531/1043] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Sep 6 04:18:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 04:18:35.222022 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c122b14-8c88-4363-b0d5-3784e037c9ca None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:18:35.296163 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc9a94c-bdb9-4417-a604-0399392f4cc7 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:18:35.328634 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:18:35.330424 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1c3732-47f0-4c50-9986-f8a175a2e3ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:18:35.331063 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1c3732-47f0-4c50-9986-f8a175a2e3ba None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:18:35.331345 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1c3732-47f0-4c50-9986-f8a175a2e3ba None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:18:35.331622 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1c3732-47f0-4c50-9986-f8a175a2e3ba None admin] PATH_INFO: `/v3/services/image` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:18:35.332206 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1c3732-47f0-4c50-9986-f8a175a2e3ba None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:18:35.333594 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1c3732-47f0-4c50-9986-f8a175a2e3ba None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:18:35.336827 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c122b14-8c88-4363-b0d5-3784e037c9ca None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:18:35.342302 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-6b1c3732-47f0-4c50-9986-f8a175a2e3ba None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 06 04:18:35.342302 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 04:18:35.342302 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 06 04:18:35.342302 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 06 04:18:35.342302 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 06 04:18:35.342302 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 06 04:18:35.342302 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 06 04:18:35.342302 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 04:18:35.342302 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 06 04:18:35.342302 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 04:18:35.342302 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 06 04:18:35.342302 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 04:18:35.342302 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 04:18:35.342302 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 04:18:35.342302 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 04:18:35.342302 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 04:18:35.342302 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 04:18:35.345933 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 04:18:35.345933 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 04:18:35.345933 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 04:18:35.345933 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 04:18:35.345933 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 04:18:35.345933 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 04:18:35.345933 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 06 04:18:35.345933 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 06 04:18:35.345933 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 06 04:18:35.345933 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 06 04:18:35.345933 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 04:18:35.345933 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 04:18:35.345933 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 04:18:35.345933 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 04:18:35.345933 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 04:18:35.345933 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 04:18:35.346979 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 04:18:35.346979 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 04:18:35.346979 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 04:18:35.346979 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 04:18:35.346979 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 04:18:35.346979 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 04:18:35.346979 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 04:18:35.346979 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 04:18:35.346979 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 04:18:35.346979 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 04:18:35.346979 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 06 04:18:35.346979 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 04:18:35.346979 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 06 04:18:35.346979 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 04:18:35.346979 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 513/1044] 10.222.0.30 () {56 vars in 1105 bytes} [Tue Sep 6 04:18:35 2022] GET /identity/v3/services/image => generated 86 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 04:18:35.350617 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1c3732-47f0-4c50-9986-f8a175a2e3ba None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:18:35.381211 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb755045-39c5-49a9-959f-03ce3aa0306d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:18:35.381952 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb755045-39c5-49a9-959f-03ce3aa0306d None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:18:35.382304 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb755045-39c5-49a9-959f-03ce3aa0306d None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:18:35.382651 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb755045-39c5-49a9-959f-03ce3aa0306d None admin] PATH_INFO: `/v3/domains` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:18:35.383364 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb755045-39c5-49a9-959f-03ce3aa0306d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:18:35.383734 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb755045-39c5-49a9-959f-03ce3aa0306d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:18:35.385272 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb755045-39c5-49a9-959f-03ce3aa0306d None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:18:35.398873 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 532/1045] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Sep 6 04:18:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 04:18:35.404209 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb755045-39c5-49a9-959f-03ce3aa0306d None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:18:35.416448 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:18:35.464250 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1c3732-47f0-4c50-9986-f8a175a2e3ba None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:18:35.500453 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099097d3-32d8-450e-abc9-b0ae5982ff09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:18:35.501106 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099097d3-32d8-450e-abc9-b0ae5982ff09 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:18:35.501403 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099097d3-32d8-450e-abc9-b0ae5982ff09 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:18:35.502138 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099097d3-32d8-450e-abc9-b0ae5982ff09 None admin] PATH_INFO: `/v3/services` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:18:35.502804 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099097d3-32d8-450e-abc9-b0ae5982ff09 None admin] RBAC: Adding query filter params (name=image) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:18:35.503112 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099097d3-32d8-450e-abc9-b0ae5982ff09 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:18:35.504488 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099097d3-32d8-450e-abc9-b0ae5982ff09 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:18:35.509383 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 514/1046] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Sep 6 04:18:35 2022] GET /identity/v3/services?name=image => generated 122 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 04:18:35.512888 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb755045-39c5-49a9-959f-03ce3aa0306d None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:18:35.514911 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099097d3-32d8-450e-abc9-b0ae5982ff09 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:18:35.551005 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc97e7a1-50c0-4224-81cb-905f16b692a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:18:35.551985 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc97e7a1-50c0-4224-81cb-905f16b692a1 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:18:35.552262 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc97e7a1-50c0-4224-81cb-905f16b692a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:18:35.552527 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc97e7a1-50c0-4224-81cb-905f16b692a1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:18:35.560559 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc97e7a1-50c0-4224-81cb-905f16b692a1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:18:35.562151 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc97e7a1-50c0-4224-81cb-905f16b692a1 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:18:35.574168 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-fc97e7a1-50c0-4224-81cb-905f16b692a1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 04:18:35.574168 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 04:18:35.574168 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 04:18:35.574168 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 04:18:35.574168 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 04:18:35.574168 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 04:18:35.574168 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 04:18:35.574168 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 04:18:35.574168 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 04:18:35.574168 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 04:18:35.574168 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 04:18:35.574168 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 04:18:35.574168 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 06 04:18:35.574168 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 06 04:18:35.574168 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 06 04:18:35.575073 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 06 04:18:35.575073 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 04:18:35.575073 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 04:18:35.575073 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 04:18:35.575073 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 04:18:35.575073 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 04:18:35.575073 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 04:18:35.575073 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 04:18:35.575073 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 04:18:35.575073 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 04:18:35.575073 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 04:18:35.575073 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 04:18:35.575073 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 04:18:35.575073 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 04:18:35.575073 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 04:18:35.575073 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 04:18:35.575073 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 04:18:35.576300 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 06 04:18:35.576300 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 06 04:18:35.576300 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 06 04:18:35.576300 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 06 04:18:35.576300 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 06 04:18:35.576300 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 06 04:18:35.576300 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 04:18:35.576300 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 04:18:35.576300 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 533/1047] 10.222.0.30 () {56 vars in 1111 bytes} [Tue Sep 6 04:18:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 04:18:35.581009 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc97e7a1-50c0-4224-81cb-905f16b692a1 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:18:35.597508 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:18:35.627256 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099097d3-32d8-450e-abc9-b0ae5982ff09 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:18:35.648509 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:18:35.662602 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7936875c-001e-4d69-9c85-6f19e3765c8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:18:35.663222 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7936875c-001e-4d69-9c85-6f19e3765c8b None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:18:35.663497 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7936875c-001e-4d69-9c85-6f19e3765c8b None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:18:35.663774 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7936875c-001e-4d69-9c85-6f19e3765c8b None admin] PATH_INFO: `/v3/services` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:18:35.664396 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7936875c-001e-4d69-9c85-6f19e3765c8b None admin] RBAC: Adding query filter params (type=image) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:18:35.664740 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7936875c-001e-4d69-9c85-6f19e3765c8b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:18:35.666127 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7936875c-001e-4d69-9c85-6f19e3765c8b None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:18:35.670871 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 515/1048] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Sep 6 04:18:35 2022] GET /identity/v3/services?type=image => generated 350 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 04:18:35.678704 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7936875c-001e-4d69-9c85-6f19e3765c8b None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:18:35.693379 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc97e7a1-50c0-4224-81cb-905f16b692a1 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:18:35.730537 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55a0417-2ac6-4f3c-b5a3-0635b151be0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:18:35.731509 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55a0417-2ac6-4f3c-b5a3-0635b151be0d None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:18:35.731998 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55a0417-2ac6-4f3c-b5a3-0635b151be0d None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:18:35.732338 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55a0417-2ac6-4f3c-b5a3-0635b151be0d None admin] PATH_INFO: `/v3/projects` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:18:35.733034 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55a0417-2ac6-4f3c-b5a3-0635b151be0d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:18:35.733357 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55a0417-2ac6-4f3c-b5a3-0635b151be0d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:18:35.734777 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55a0417-2ac6-4f3c-b5a3-0635b151be0d None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:18:35.747638 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 534/1049] 10.222.0.30 () {56 vars in 1148 bytes} [Tue Sep 6 04:18:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 04:18:35.754403 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55a0417-2ac6-4f3c-b5a3-0635b151be0d None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:18:35.784775 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7936875c-001e-4d69-9c85-6f19e3765c8b None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:18:35.821196 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b39360-360b-4ec0-aef5-0c132a2117ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:18:35.821854 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b39360-360b-4ec0-aef5-0c132a2117ad None admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:18:35.822133 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b39360-360b-4ec0-aef5-0c132a2117ad None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:18:35.822403 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b39360-360b-4ec0-aef5-0c132a2117ad None admin] PATH_INFO: `/v3/endpoints` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:18:35.823061 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b39360-360b-4ec0-aef5-0c132a2117ad None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:18:35.824409 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b39360-360b-4ec0-aef5-0c132a2117ad None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:18:35.834828 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:18:35.859662 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55a0417-2ac6-4f3c-b5a3-0635b151be0d None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:18:35.892779 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a21f4f4-c277-4a0f-b931-d289d6795a39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:18:35.893497 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a21f4f4-c277-4a0f-b931-d289d6795a39 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:18:35.893781 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a21f4f4-c277-4a0f-b931-d289d6795a39 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:18:35.894047 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a21f4f4-c277-4a0f-b931-d289d6795a39 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:18:35.894763 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a21f4f4-c277-4a0f-b931-d289d6795a39 None admin] RBAC: Adding query filter params (role.id=356631f6a0064bc695b78e3759d55d6a, scope.project.id=b707733ce65c42edbc1fd95ad0a1c06c, user.id=9db138b77afa4b068244b8ef715404fe) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:18:35.895062 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a21f4f4-c277-4a0f-b931-d289d6795a39 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:18:35.896496 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a21f4f4-c277-4a0f-b931-d289d6795a39 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:18:35.903462 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 535/1050] 10.222.0.30 () {56 vars in 1374 bytes} [Tue Sep 6 04:18:35 2022] GET /identity/v3/role_assignments?user.id=9db138b77afa4b068244b8ef715404fe&scope.project.id=b707733ce65c42edbc1fd95ad0a1c06c&role.id=356631f6a0064bc695b78e3759d55d6a => generated 259 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:18:35.945336 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 516/1051] 10.222.0.30 () {60 vars in 1144 bytes} [Tue Sep 6 04:18:35 2022] POST /identity/v3/endpoints => generated 327 bytes in 258 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 04:18:36.102912 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83682b30-117b-49b1-9e73-e04d2afc472b None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:18:36.103288 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83682b30-117b-49b1-9e73-e04d2afc472b None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:18:36.103474 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83682b30-117b-49b1-9e73-e04d2afc472b None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:18:36.104818 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 536/1052] 10.222.0.30 () {54 vars in 927 bytes} [Tue Sep 6 04:18:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 04:18:36.110268 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b462d28a-73d8-42e5-b90b-4f9a74e4b0a0 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:18:36.110560 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b462d28a-73d8-42e5-b90b-4f9a74e4b0a0 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:18:36.111375 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b462d28a-73d8-42e5-b90b-4f9a74e4b0a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:18:36.130790 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-b462d28a-73d8-42e5-b90b-4f9a74e4b0a0 None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:18:36.132007 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-b462d28a-73d8-42e5-b90b-4f9a74e4b0a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:18:36.167017 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:18:36.206120 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 517/1053] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Sep 6 04:18:36 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 04:18:36.483754 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83682b30-117b-49b1-9e73-e04d2afc472b None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:18:36.587066 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83682b30-117b-49b1-9e73-e04d2afc472b None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:18:36.622159 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b42bf7-63e1-4df2-ab41-481cf76e3e4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:18:36.622787 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b42bf7-63e1-4df2-ab41-481cf76e3e4b None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:18:36.623064 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b42bf7-63e1-4df2-ab41-481cf76e3e4b None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:18:36.623335 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b42bf7-63e1-4df2-ab41-481cf76e3e4b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:18:36.630416 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:18:36.634681 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b42bf7-63e1-4df2-ab41-481cf76e3e4b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 04:18:36.639566 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-42b42bf7-63e1-4df2-ab41-481cf76e3e4b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 04:18:36.639566 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 04:18:36.639566 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 04:18:36.639566 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 04:18:36.639566 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 04:18:36.639566 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 04:18:36.639566 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 04:18:36.639566 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 04:18:36.639566 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 04:18:36.639566 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 04:18:36.639566 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 04:18:36.639566 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 04:18:36.639566 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 04:18:36.639566 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 04:18:36.639566 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 04:18:36.639566 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 04:18:36.643915 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 04:18:36.643915 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 04:18:36.643915 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 04:18:36.643915 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 04:18:36.643915 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 04:18:36.643915 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 04:18:36.643915 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 04:18:36.643915 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 04:18:36.643915 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 04:18:36.643915 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 04:18:36.643915 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 04:18:36.643915 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 04:18:36.643915 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 04:18:36.643915 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 04:18:36.643915 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 04:18:36.643915 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 04:18:36.644788 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 04:18:36.644788 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 04:18:36.644788 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 04:18:36.644788 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 04:18:36.644788 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 04:18:36.644788 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 04:18:36.644788 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 04:18:36.644788 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 04:18:36.644788 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 04:18:36.644788 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 04:18:36.644788 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 04:18:36.644788 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 04:18:36.644788 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 537/1054] 10.222.0.30 () {56 vars in 1096 bytes} [Tue Sep 6 04:18:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 04:18:36.647273 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b462d28a-73d8-42e5-b90b-4f9a74e4b0a0 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:18:36.752328 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b462d28a-73d8-42e5-b90b-4f9a74e4b0a0 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:18:36.787927 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12fb0fd3-c0b8-4859-b670-607c4eb959b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:18:36.788543 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fb0fd3-c0b8-4859-b670-607c4eb959b7 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:18:36.788823 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fb0fd3-c0b8-4859-b670-607c4eb959b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:18:36.789092 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fb0fd3-c0b8-4859-b670-607c4eb959b7 None admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:18:36.789782 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12fb0fd3-c0b8-4859-b670-607c4eb959b7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:18:36.790085 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12fb0fd3-c0b8-4859-b670-607c4eb959b7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:18:36.791449 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12fb0fd3-c0b8-4859-b670-607c4eb959b7 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:18:36.800024 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 518/1055] 10.222.0.30 () {56 vars in 1099 bytes} [Tue Sep 6 04:18:36 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:18:36.805729 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b42bf7-63e1-4df2-ab41-481cf76e3e4b None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:18:36.906921 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b42bf7-63e1-4df2-ab41-481cf76e3e4b None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:18:36.941132 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7374170d-f4f6-497d-bc12-572d8b169bc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:18:36.941797 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7374170d-f4f6-497d-bc12-572d8b169bc7 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:18:36.942186 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7374170d-f4f6-497d-bc12-572d8b169bc7 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:18:36.942606 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7374170d-f4f6-497d-bc12-572d8b169bc7 None admin] PATH_INFO: `/v3/users/8768ff5946b14186a24648064c3975ec` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:18:36.956829 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7374170d-f4f6-497d-bc12-572d8b169bc7 None admin] RBAC: Authorizing `identity:get_user(user_id=8768ff5946b14186a24648064c3975ec)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:18:36.958301 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7374170d-f4f6-497d-bc12-572d8b169bc7 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:18:36.964756 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 538/1056] 10.222.0.30 () {56 vars in 1177 bytes} [Tue Sep 6 04:18:36 2022] GET /identity/v3/users/8768ff5946b14186a24648064c3975ec => generated 285 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:18:36.965980 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12fb0fd3-c0b8-4859-b670-607c4eb959b7 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:18:36.997402 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:18:37.073362 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12fb0fd3-c0b8-4859-b670-607c4eb959b7 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:18:37.107071 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cadbd866-229f-42ed-9e64-1abc85bec313 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:18:37.107688 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadbd866-229f-42ed-9e64-1abc85bec313 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:18:37.107961 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadbd866-229f-42ed-9e64-1abc85bec313 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:18:37.108230 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadbd866-229f-42ed-9e64-1abc85bec313 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:18:37.184030 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cadbd866-229f-42ed-9e64-1abc85bec313 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:18:37.185425 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cadbd866-229f-42ed-9e64-1abc85bec313 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:18:37.212354 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:18:37.291609 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 519/1057] 10.222.0.30 () {58 vars in 1282 bytes} [Tue Sep 6 04:18:36 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 06 04:18:37.293198 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7374170d-f4f6-497d-bc12-572d8b169bc7 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:18:37.350837 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:18:37.402453 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7374170d-f4f6-497d-bc12-572d8b169bc7 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:18:37.438212 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6640656-d0bb-4eca-b086-fae56f67979a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:18:37.438831 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6640656-d0bb-4eca-b086-fae56f67979a None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:18:37.440015 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6640656-d0bb-4eca-b086-fae56f67979a None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:18:37.440460 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6640656-d0bb-4eca-b086-fae56f67979a None admin] PATH_INFO: `/v3/projects/1ac1bf587f2b4700b21dd5043c4bd7c2` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:18:37.441641 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6640656-d0bb-4eca-b086-fae56f67979a None admin] RBAC: Authorizing `identity:get_project(project_id=1ac1bf587f2b4700b21dd5043c4bd7c2)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:18:37.443148 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6640656-d0bb-4eca-b086-fae56f67979a None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:18:37.445011 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 539/1058] 10.222.0.30 () {56 vars in 1186 bytes} [Tue Sep 6 04:18:37 2022] GET /identity/v3/projects/1ac1bf587f2b4700b21dd5043c4bd7c2 => generated 311 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 04:18:37.450910 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cadbd866-229f-42ed-9e64-1abc85bec313 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:18:37.556845 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cadbd866-229f-42ed-9e64-1abc85bec313 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:18:37.584900 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:18:37.593121 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6737e062-fc35-4bf5-a853-f0878a014511 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:18:37.593777 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6737e062-fc35-4bf5-a853-f0878a014511 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:18:37.594055 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6737e062-fc35-4bf5-a853-f0878a014511 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:18:37.594326 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6737e062-fc35-4bf5-a853-f0878a014511 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:18:37.595095 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6737e062-fc35-4bf5-a853-f0878a014511 None admin] RBAC: Adding query filter params (role.id=ea4c814713b149108e488de855b025e4, scope.project.id=1ac1bf587f2b4700b21dd5043c4bd7c2, user.id=8768ff5946b14186a24648064c3975ec) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:18:37.595539 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6737e062-fc35-4bf5-a853-f0878a014511 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:18:37.597234 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6737e062-fc35-4bf5-a853-f0878a014511 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:18:37.609340 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 520/1059] 10.222.0.30 () {56 vars in 1374 bytes} [Tue Sep 6 04:18:37 2022] GET /identity/v3/role_assignments?user.id=8768ff5946b14186a24648064c3975ec&scope.project.id=1ac1bf587f2b4700b21dd5043c4bd7c2&role.id=ea4c814713b149108e488de855b025e4 => generated 259 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:18:37.916329 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcd29cc-8ab5-4c19-a899-21cdf697ae6f None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:18:37.916844 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcd29cc-8ab5-4c19-a899-21cdf697ae6f None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:18:37.916957 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcd29cc-8ab5-4c19-a899-21cdf697ae6f None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:18:37.918296 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 540/1060] 10.222.0.30 () {54 vars in 927 bytes} [Tue Sep 6 04:18:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 04:18:37.919753 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c7195b-3f3b-409c-b0b2-d7d7e640c5b6 None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:18:37.920324 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c7195b-3f3b-409c-b0b2-d7d7e640c5b6 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:18:37.920324 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c7195b-3f3b-409c-b0b2-d7d7e640c5b6 None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:18:37.921645 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 521/1061] 10.222.0.30 () {54 vars in 927 bytes} [Tue Sep 6 04:18:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 04:18:37.923388 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c1c125-1ccf-4cee-884b-47674409c925 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:18:37.923757 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c1c125-1ccf-4cee-884b-47674409c925 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:18:37.924041 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c1c125-1ccf-4cee-884b-47674409c925 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:18:37.928171 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc1eb4c-2cf3-4696-a64b-f4e2a75f2e08 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:18:37.928482 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc1eb4c-2cf3-4696-a64b-f4e2a75f2e08 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:18:37.928847 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc1eb4c-2cf3-4696-a64b-f4e2a75f2e08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:18:37.942840 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-01c1c125-1ccf-4cee-884b-47674409c925 None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:18:37.944045 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-01c1c125-1ccf-4cee-884b-47674409c925 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:18:37.948862 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-9fc1eb4c-2cf3-4696-a64b-f4e2a75f2e08 None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:18:37.950062 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-9fc1eb4c-2cf3-4696-a64b-f4e2a75f2e08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:18:38.028289 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 522/1062] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Sep 6 04:18:37 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 06 04:18:38.037351 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 541/1063] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Sep 6 04:18:37 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 04:18:38.307358 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc1eb4c-2cf3-4696-a64b-f4e2a75f2e08 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:18:38.357387 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c1c125-1ccf-4cee-884b-47674409c925 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:18:38.434230 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:18:38.442591 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc1eb4c-2cf3-4696-a64b-f4e2a75f2e08 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:18:38.471075 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:18:38.483822 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694fe117-92e3-400b-b117-9c4547a765de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:18:38.484647 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694fe117-92e3-400b-b117-9c4547a765de None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:18:38.484932 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694fe117-92e3-400b-b117-9c4547a765de None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:18:38.486817 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694fe117-92e3-400b-b117-9c4547a765de None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:18:38.493543 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c1c125-1ccf-4cee-884b-47674409c925 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:18:38.497396 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694fe117-92e3-400b-b117-9c4547a765de None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 04:18:38.502767 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-694fe117-92e3-400b-b117-9c4547a765de None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 06 04:18:38.502767 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 04:18:38.502767 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 04:18:38.502767 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 04:18:38.502767 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 04:18:38.502767 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 04:18:38.502767 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 04:18:38.502767 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 04:18:38.502767 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 04:18:38.502767 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 04:18:38.502767 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 04:18:38.502767 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 04:18:38.502767 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 04:18:38.502767 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 04:18:38.502767 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 04:18:38.502767 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 04:18:38.504372 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 04:18:38.504372 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 04:18:38.504372 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 04:18:38.504372 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 04:18:38.504372 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 04:18:38.504372 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 04:18:38.504372 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 04:18:38.504372 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 04:18:38.504372 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 04:18:38.504372 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 04:18:38.504372 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 04:18:38.504372 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 04:18:38.504372 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 04:18:38.504372 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 04:18:38.504372 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 04:18:38.504372 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 04:18:38.506466 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 04:18:38.506466 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 04:18:38.506466 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 04:18:38.506466 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 04:18:38.506466 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 04:18:38.506466 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 04:18:38.506466 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 04:18:38.506466 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 04:18:38.506466 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 04:18:38.506466 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 04:18:38.506466 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 06 04:18:38.506466 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 04:18:38.509272 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 523/1064] 10.222.0.30 () {56 vars in 1120 bytes} [Tue Sep 6 04:18:38 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 04:18:38.511498 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694fe117-92e3-400b-b117-9c4547a765de None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:18:38.548951 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6561a789-cf59-4d2f-b36a-df6aadf38177 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:18:38.549615 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6561a789-cf59-4d2f-b36a-df6aadf38177 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:18:38.549897 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6561a789-cf59-4d2f-b36a-df6aadf38177 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:18:38.550169 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6561a789-cf59-4d2f-b36a-df6aadf38177 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:18:38.638427 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6561a789-cf59-4d2f-b36a-df6aadf38177 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:18:38.639957 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6561a789-cf59-4d2f-b36a-df6aadf38177 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:18:38.650950 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694fe117-92e3-400b-b117-9c4547a765de None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:18:38.700620 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae70ed6-d2a5-4592-a5bf-acd98e2a296a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:18:38.701225 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae70ed6-d2a5-4592-a5bf-acd98e2a296a None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:18:38.701694 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae70ed6-d2a5-4592-a5bf-acd98e2a296a None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:18:38.701802 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae70ed6-d2a5-4592-a5bf-acd98e2a296a None admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:18:38.702477 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae70ed6-d2a5-4592-a5bf-acd98e2a296a None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:18:38.702779 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae70ed6-d2a5-4592-a5bf-acd98e2a296a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:18:38.704173 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae70ed6-d2a5-4592-a5bf-acd98e2a296a None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:18:38.708052 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:18:38.714869 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 524/1065] 10.222.0.30 () {56 vars in 1115 bytes} [Tue Sep 6 04:18:38 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:18:38.721572 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae70ed6-d2a5-4592-a5bf-acd98e2a296a None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:18:38.772662 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 542/1066] 10.222.0.30 () {58 vars in 1282 bytes} [Tue Sep 6 04:18:38 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 417 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 06 04:18:38.780277 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6561a789-cf59-4d2f-b36a-df6aadf38177 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:18:38.797495 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:18:38.875251 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae70ed6-d2a5-4592-a5bf-acd98e2a296a None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:18:38.904701 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6561a789-cf59-4d2f-b36a-df6aadf38177 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:18:38.912264 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6114bb8-4a2c-48f1-9950-fff7d5aa785a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:18:38.912906 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6114bb8-4a2c-48f1-9950-fff7d5aa785a None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:18:38.913187 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6114bb8-4a2c-48f1-9950-fff7d5aa785a None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:18:38.913493 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6114bb8-4a2c-48f1-9950-fff7d5aa785a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:18:38.924101 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6114bb8-4a2c-48f1-9950-fff7d5aa785a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:18:38.925568 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6114bb8-4a2c-48f1-9950-fff7d5aa785a None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:18:38.939295 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-f6114bb8-4a2c-48f1-9950-fff7d5aa785a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 04:18:38.939295 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 04:18:38.939295 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 04:18:38.939295 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 04:18:38.939295 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 04:18:38.939295 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 04:18:38.939295 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 04:18:38.939295 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 04:18:38.939295 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 04:18:38.939295 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 04:18:38.939295 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 04:18:38.939295 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 04:18:38.939295 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 04:18:38.939295 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 04:18:38.939295 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 04:18:38.939295 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 04:18:38.940299 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 04:18:38.940299 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 04:18:38.940299 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 04:18:38.940299 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 04:18:38.940299 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 04:18:38.940299 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 04:18:38.940299 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 04:18:38.940299 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 04:18:38.940299 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 04:18:38.940299 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 04:18:38.940299 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 04:18:38.940299 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 04:18:38.940299 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 04:18:38.940299 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 04:18:38.940299 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 04:18:38.940299 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 04:18:38.940299 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 04:18:38.941710 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 04:18:38.941710 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 04:18:38.941710 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 04:18:38.941710 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 525/1067] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Sep 6 04:18:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 04:18:38.946071 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6114bb8-4a2c-48f1-9950-fff7d5aa785a None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:18:38.946512 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df89b7b-5696-4d50-adc5-4e75a294e158 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:18:38.947641 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df89b7b-5696-4d50-adc5-4e75a294e158 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:18:38.948072 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df89b7b-5696-4d50-adc5-4e75a294e158 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:18:38.948636 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df89b7b-5696-4d50-adc5-4e75a294e158 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:18:38.960562 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df89b7b-5696-4d50-adc5-4e75a294e158 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:18:38.962526 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df89b7b-5696-4d50-adc5-4e75a294e158 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:18:38.979013 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-3df89b7b-5696-4d50-adc5-4e75a294e158 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 04:18:38.979013 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 04:18:38.979013 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 04:18:38.979013 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 04:18:38.979013 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 04:18:38.979013 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 04:18:38.979013 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 04:18:38.979013 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 04:18:38.979013 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 04:18:38.979013 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 04:18:38.979013 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 04:18:38.979013 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 04:18:38.979013 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 04:18:38.979013 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 04:18:38.979013 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 04:18:38.979013 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 04:18:38.980164 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 04:18:38.980164 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 04:18:38.980164 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 04:18:38.980164 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 04:18:38.980164 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 04:18:38.980164 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 04:18:38.980164 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 04:18:38.980164 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 04:18:38.980164 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 04:18:38.980164 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 04:18:38.980164 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 04:18:38.980164 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 04:18:38.980164 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 04:18:38.980164 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 04:18:38.980164 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 04:18:38.980164 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 04:18:38.980164 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 04:18:38.981283 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 04:18:38.981283 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 04:18:38.981283 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 04:18:38.982748 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 543/1068] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Sep 6 04:18:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 04:18:38.988638 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df89b7b-5696-4d50-adc5-4e75a294e158 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:18:38.998511 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:18:39.047417 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:18:39.054359 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6114bb8-4a2c-48f1-9950-fff7d5aa785a None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:18:39.090581 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df89b7b-5696-4d50-adc5-4e75a294e158 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:18:39.092548 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f51157b-f41b-48f7-a918-486188bd1ce4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:18:39.093276 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f51157b-f41b-48f7-a918-486188bd1ce4 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:18:39.093593 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f51157b-f41b-48f7-a918-486188bd1ce4 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:18:39.093891 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f51157b-f41b-48f7-a918-486188bd1ce4 None admin] PATH_INFO: `/v3/domains` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:18:39.094538 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f51157b-f41b-48f7-a918-486188bd1ce4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:18:39.094841 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f51157b-f41b-48f7-a918-486188bd1ce4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:18:39.096779 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f51157b-f41b-48f7-a918-486188bd1ce4 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:18:39.114048 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 526/1069] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Sep 6 04:18:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 04:18:39.115117 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f51157b-f41b-48f7-a918-486188bd1ce4 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:18:39.129398 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e56ee92-fcd8-4df2-b12f-133161a1da29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:18:39.130013 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e56ee92-fcd8-4df2-b12f-133161a1da29 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:18:39.130329 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e56ee92-fcd8-4df2-b12f-133161a1da29 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:18:39.130561 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e56ee92-fcd8-4df2-b12f-133161a1da29 None admin] PATH_INFO: `/v3/domains` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:18:39.131208 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e56ee92-fcd8-4df2-b12f-133161a1da29 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:18:39.131516 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e56ee92-fcd8-4df2-b12f-133161a1da29 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:18:39.132913 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e56ee92-fcd8-4df2-b12f-133161a1da29 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:18:39.149356 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 544/1070] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Sep 6 04:18:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 04:18:39.230854 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f51157b-f41b-48f7-a918-486188bd1ce4 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:18:39.270266 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9bb0ca4-850d-430e-839a-d57e5b440110 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:18:39.270926 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bb0ca4-850d-430e-839a-d57e5b440110 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:18:39.271203 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bb0ca4-850d-430e-839a-d57e5b440110 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:18:39.271475 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bb0ca4-850d-430e-839a-d57e5b440110 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:18:39.279469 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9bb0ca4-850d-430e-839a-d57e5b440110 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:18:39.280921 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9bb0ca4-850d-430e-839a-d57e5b440110 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:18:39.293705 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-e9bb0ca4-850d-430e-839a-d57e5b440110 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 06 04:18:39.293705 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 04:18:39.293705 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 04:18:39.293705 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 04:18:39.293705 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 04:18:39.293705 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 04:18:39.293705 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 04:18:39.293705 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 04:18:39.293705 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 04:18:39.293705 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 04:18:39.293705 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 04:18:39.293705 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 04:18:39.293705 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 06 04:18:39.293705 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 06 04:18:39.293705 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 06 04:18:39.293705 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 06 04:18:39.294726 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 04:18:39.294726 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 04:18:39.294726 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 06 04:18:39.294726 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 04:18:39.294726 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 06 04:18:39.294726 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 04:18:39.294726 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 04:18:39.294726 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 04:18:39.294726 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 04:18:39.294726 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 04:18:39.294726 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 04:18:39.294726 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 04:18:39.294726 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 04:18:39.294726 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 04:18:39.294726 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 04:18:39.294726 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 04:18:39.294726 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 04:18:39.294726 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 04:18:39.295723 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 04:18:39.295723 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 04:18:39.295723 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 06 04:18:39.295723 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 06 04:18:39.295723 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 06 04:18:39.295723 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 06 04:18:39.295723 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 06 04:18:39.295723 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 06 04:18:39.295723 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 06 04:18:39.295723 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 04:18:39.295723 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 527/1071] 10.222.0.30 () {56 vars in 1093 bytes} [Tue Sep 6 04:18:39 2022] GET /identity/v3/users/nova => generated 82 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 04:18:39.302751 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e56ee92-fcd8-4df2-b12f-133161a1da29 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:18:39.375152 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:18:39.415700 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e56ee92-fcd8-4df2-b12f-133161a1da29 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:18:39.452965 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0a1646-6ae1-4b69-a229-797a919316c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:18:39.453620 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0a1646-6ae1-4b69-a229-797a919316c2 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:18:39.453894 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0a1646-6ae1-4b69-a229-797a919316c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:18:39.454162 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0a1646-6ae1-4b69-a229-797a919316c2 None admin] PATH_INFO: `/v3/users` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:18:39.455066 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0a1646-6ae1-4b69-a229-797a919316c2 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:18:39.455369 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0a1646-6ae1-4b69-a229-797a919316c2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:18:39.456803 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0a1646-6ae1-4b69-a229-797a919316c2 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:18:39.475219 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 545/1072] 10.222.0.30 () {56 vars in 1133 bytes} [Tue Sep 6 04:18:39 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:18:39.481083 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9bb0ca4-850d-430e-839a-d57e5b440110 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:18:39.500510 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:18:39.581124 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9bb0ca4-850d-430e-839a-d57e5b440110 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:18:39.613271 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af6a88e-409c-463f-aad4-9e46cef875cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:18:39.613911 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af6a88e-409c-463f-aad4-9e46cef875cf None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:18:39.614185 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af6a88e-409c-463f-aad4-9e46cef875cf None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:18:39.614453 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af6a88e-409c-463f-aad4-9e46cef875cf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:18:39.624518 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af6a88e-409c-463f-aad4-9e46cef875cf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:18:39.625977 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af6a88e-409c-463f-aad4-9e46cef875cf None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:18:39.640504 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-4af6a88e-409c-463f-aad4-9e46cef875cf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 04:18:39.640504 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 04:18:39.640504 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 04:18:39.640504 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 04:18:39.640504 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 04:18:39.640504 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 04:18:39.640504 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 04:18:39.640504 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 04:18:39.640504 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 04:18:39.640504 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 04:18:39.640504 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 04:18:39.640504 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 04:18:39.640504 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 04:18:39.640504 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 04:18:39.640504 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 04:18:39.640504 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 04:18:39.641652 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 04:18:39.641652 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 04:18:39.641652 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 04:18:39.641652 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 04:18:39.641652 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 04:18:39.641652 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 04:18:39.641652 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 04:18:39.641652 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 04:18:39.641652 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 04:18:39.641652 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 04:18:39.641652 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 04:18:39.641652 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 04:18:39.641652 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 04:18:39.641652 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 04:18:39.641652 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 04:18:39.641652 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 04:18:39.641652 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 04:18:39.642977 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 04:18:39.642977 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 04:18:39.642977 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 04:18:39.642977 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 528/1073] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Sep 6 04:18:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 04:18:39.647522 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0a1646-6ae1-4b69-a229-797a919316c2 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:18:39.669569 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14eeba93-204f-43ba-94e1-86f9380f198a None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:18:39.669851 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14eeba93-204f-43ba-94e1-86f9380f198a None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:18:39.670120 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14eeba93-204f-43ba-94e1-86f9380f198a None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:18:39.671446 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 529/1074] 10.222.0.30 () {54 vars in 927 bytes} [Tue Sep 6 04:18:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 06 04:18:39.677168 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1158dc-bd1a-4775-a55c-ca501e9821cb None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:18:39.677494 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1158dc-bd1a-4775-a55c-ca501e9821cb None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:18:39.677776 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1158dc-bd1a-4775-a55c-ca501e9821cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:18:39.696152 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-4a1158dc-bd1a-4775-a55c-ca501e9821cb None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:18:39.697379 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-4a1158dc-bd1a-4775-a55c-ca501e9821cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:18:39.750125 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0a1646-6ae1-4b69-a229-797a919316c2 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:18:39.759451 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:18:39.765168 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 530/1075] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Sep 6 04:18:39 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 06 04:18:39.774271 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:18:39.784698 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3e84e7-0f7d-4628-8f97-ae1b34856f85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:18:39.785328 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3e84e7-0f7d-4628-8f97-ae1b34856f85 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:18:39.785618 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3e84e7-0f7d-4628-8f97-ae1b34856f85 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:18:39.785891 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3e84e7-0f7d-4628-8f97-ae1b34856f85 None admin] PATH_INFO: `/v3/domains` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:18:39.786535 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3e84e7-0f7d-4628-8f97-ae1b34856f85 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:18:39.786838 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3e84e7-0f7d-4628-8f97-ae1b34856f85 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:18:39.788214 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3e84e7-0f7d-4628-8f97-ae1b34856f85 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:18:39.799716 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 546/1076] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Sep 6 04:18:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:18:39.805519 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1158dc-bd1a-4775-a55c-ca501e9821cb None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:18:39.901760 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1158dc-bd1a-4775-a55c-ca501e9821cb None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:18:39.934365 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c13fcd-44be-44a7-ac62-d81fb95fd23f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:18:39.934981 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c13fcd-44be-44a7-ac62-d81fb95fd23f None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:18:39.935244 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c13fcd-44be-44a7-ac62-d81fb95fd23f None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:18:39.935512 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c13fcd-44be-44a7-ac62-d81fb95fd23f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:18:39.942529 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c13fcd-44be-44a7-ac62-d81fb95fd23f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:18:39.943946 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c13fcd-44be-44a7-ac62-d81fb95fd23f None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:18:39.954965 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-07c13fcd-44be-44a7-ac62-d81fb95fd23f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 04:18:39.954965 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 04:18:39.954965 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 04:18:39.954965 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 04:18:39.954965 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 04:18:39.954965 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 04:18:39.954965 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 04:18:39.954965 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 04:18:39.954965 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 04:18:39.954965 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 04:18:39.954965 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 04:18:39.954965 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 04:18:39.954965 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 06 04:18:39.954965 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 06 04:18:39.954965 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 06 04:18:39.956007 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 06 04:18:39.956007 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 04:18:39.956007 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 04:18:39.956007 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 04:18:39.956007 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 04:18:39.956007 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 04:18:39.956007 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 04:18:39.956007 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 04:18:39.956007 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 04:18:39.956007 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 04:18:39.956007 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 04:18:39.956007 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 04:18:39.956007 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 04:18:39.956007 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 04:18:39.956007 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 04:18:39.956007 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 04:18:39.956007 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 04:18:39.957096 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 06 04:18:39.957096 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 06 04:18:39.957096 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 06 04:18:39.957096 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 06 04:18:39.957096 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 06 04:18:39.957096 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 06 04:18:39.957096 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 04:18:39.957096 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 04:18:39.960793 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 531/1077] 10.222.0.30 () {56 vars in 1111 bytes} [Tue Sep 6 04:18:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 04:18:39.962019 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3e84e7-0f7d-4628-8f97-ae1b34856f85 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:18:40.037222 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c13fcd-44be-44a7-ac62-d81fb95fd23f None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:18:40.069105 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3e84e7-0f7d-4628-8f97-ae1b34856f85 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:18:40.096413 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:18:40.103390 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abdc8813-53b8-46b8-8435-c2bef23de0ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:18:40.103996 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdc8813-53b8-46b8-8435-c2bef23de0ab None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:18:40.104277 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdc8813-53b8-46b8-8435-c2bef23de0ab None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:18:40.104548 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdc8813-53b8-46b8-8435-c2bef23de0ab None admin] PATH_INFO: `/v3/projects` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:18:40.105235 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abdc8813-53b8-46b8-8435-c2bef23de0ab None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:18:40.105569 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abdc8813-53b8-46b8-8435-c2bef23de0ab None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:18:40.107702 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abdc8813-53b8-46b8-8435-c2bef23de0ab None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:18:40.119416 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 547/1078] 10.222.0.30 () {56 vars in 1148 bytes} [Tue Sep 6 04:18:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:18:40.125401 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abdc8813-53b8-46b8-8435-c2bef23de0ab None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:18:40.138543 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c13fcd-44be-44a7-ac62-d81fb95fd23f None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:18:40.170513 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:18:40.171865 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490facde-9efb-4d8c-a58f-12b0d37ffac8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:18:40.172493 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490facde-9efb-4d8c-a58f-12b0d37ffac8 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:18:40.172771 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490facde-9efb-4d8c-a58f-12b0d37ffac8 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:18:40.173044 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490facde-9efb-4d8c-a58f-12b0d37ffac8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:18:40.182992 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-490facde-9efb-4d8c-a58f-12b0d37ffac8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 04:18:40.188149 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-490facde-9efb-4d8c-a58f-12b0d37ffac8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 04:18:40.188149 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 04:18:40.188149 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 04:18:40.188149 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 04:18:40.188149 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 04:18:40.188149 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 04:18:40.188149 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 04:18:40.188149 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 04:18:40.188149 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 04:18:40.188149 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 04:18:40.188149 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 04:18:40.188149 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 04:18:40.188149 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 04:18:40.188149 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 04:18:40.188149 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 04:18:40.188149 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 04:18:40.189960 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 04:18:40.189960 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 04:18:40.189960 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 04:18:40.189960 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 04:18:40.189960 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 04:18:40.189960 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 04:18:40.189960 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 04:18:40.189960 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 04:18:40.189960 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 04:18:40.189960 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 04:18:40.189960 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 04:18:40.189960 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 04:18:40.189960 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 04:18:40.189960 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 04:18:40.189960 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 04:18:40.189960 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 04:18:40.190815 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 04:18:40.190815 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 04:18:40.190815 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 04:18:40.190815 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 04:18:40.190815 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 04:18:40.190815 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 04:18:40.190815 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 04:18:40.190815 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 04:18:40.190815 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 04:18:40.190815 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 04:18:40.190815 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 04:18:40.190815 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 04:18:40.190815 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 532/1079] 10.222.0.30 () {56 vars in 1096 bytes} [Tue Sep 6 04:18:40 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 04:18:40.195794 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490facde-9efb-4d8c-a58f-12b0d37ffac8 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:18:40.224898 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abdc8813-53b8-46b8-8435-c2bef23de0ab None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:18:40.260470 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38387b1c-04c0-4833-8e13-aba7113bd4ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:18:40.261121 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38387b1c-04c0-4833-8e13-aba7113bd4ed None admin] REQUEST_METHOD: `PUT` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:18:40.261432 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38387b1c-04c0-4833-8e13-aba7113bd4ed None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:18:40.261706 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38387b1c-04c0-4833-8e13-aba7113bd4ed None admin] PATH_INFO: `/v3/projects/b707733ce65c42edbc1fd95ad0a1c06c/users/9db138b77afa4b068244b8ef715404fe/roles/356631f6a0064bc695b78e3759d55d6a` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:18:40.263867 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38387b1c-04c0-4833-8e13-aba7113bd4ed None admin] RBAC: Authorizing `identity:create_grant(project_id=b707733ce65c42edbc1fd95ad0a1c06c, user_id=9db138b77afa4b068244b8ef715404fe, role_id=356631f6a0064bc695b78e3759d55d6a)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:18:40.265292 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38387b1c-04c0-4833-8e13-aba7113bd4ed None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:18:40.296781 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490facde-9efb-4d8c-a58f-12b0d37ffac8 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:18:40.313380 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:18:40.330157 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24dbab6-9ae7-400b-83b5-f34440f2f43a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:18:40.330751 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24dbab6-9ae7-400b-83b5-f34440f2f43a None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:18:40.331033 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24dbab6-9ae7-400b-83b5-f34440f2f43a None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:18:40.331290 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24dbab6-9ae7-400b-83b5-f34440f2f43a None admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:18:40.331936 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24dbab6-9ae7-400b-83b5-f34440f2f43a None admin] RBAC: Adding query filter params (name=admin) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:18:40.332238 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24dbab6-9ae7-400b-83b5-f34440f2f43a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:18:40.333655 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24dbab6-9ae7-400b-83b5-f34440f2f43a None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:18:40.334959 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 548/1080] 10.222.0.30 () {58 vars in 1439 bytes} [Tue Sep 6 04:18:40 2022] PUT /identity/v3/projects/b707733ce65c42edbc1fd95ad0a1c06c/users/9db138b77afa4b068244b8ef715404fe/roles/356631f6a0064bc695b78e3759d55d6a => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 04:18:40.342463 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 533/1081] 10.222.0.30 () {56 vars in 1099 bytes} [Tue Sep 6 04:18:40 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 04:18:40.349057 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38387b1c-04c0-4833-8e13-aba7113bd4ed None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:18:40.461905 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38387b1c-04c0-4833-8e13-aba7113bd4ed None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:18:40.464359 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:18:40.499299 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375af321-324c-453d-b9cf-026d29d94522 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:18:40.500135 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375af321-324c-453d-b9cf-026d29d94522 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:18:40.500586 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375af321-324c-453d-b9cf-026d29d94522 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:18:40.502141 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375af321-324c-453d-b9cf-026d29d94522 None admin] PATH_INFO: `/v3/users/8768ff5946b14186a24648064c3975ec` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:18:40.503689 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375af321-324c-453d-b9cf-026d29d94522 None admin] RBAC: Authorizing `identity:get_user(user_id=8768ff5946b14186a24648064c3975ec)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:18:40.505128 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375af321-324c-453d-b9cf-026d29d94522 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:18:40.506788 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 549/1082] 10.222.0.30 () {56 vars in 1177 bytes} [Tue Sep 6 04:18:40 2022] GET /identity/v3/users/8768ff5946b14186a24648064c3975ec => generated 285 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:18:40.512614 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24dbab6-9ae7-400b-83b5-f34440f2f43a None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:18:40.611794 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24dbab6-9ae7-400b-83b5-f34440f2f43a None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:18:40.646141 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23dd11ec-4378-4f66-bb34-4c2fc3795909 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:18:40.646799 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23dd11ec-4378-4f66-bb34-4c2fc3795909 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:18:40.647073 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23dd11ec-4378-4f66-bb34-4c2fc3795909 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:18:40.647345 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23dd11ec-4378-4f66-bb34-4c2fc3795909 None admin] PATH_INFO: `/v3/projects/1ac1bf587f2b4700b21dd5043c4bd7c2` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:18:40.648470 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23dd11ec-4378-4f66-bb34-4c2fc3795909 None admin] RBAC: Authorizing `identity:get_project(project_id=1ac1bf587f2b4700b21dd5043c4bd7c2)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:18:40.649921 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23dd11ec-4378-4f66-bb34-4c2fc3795909 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:18:40.651539 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 534/1083] 10.222.0.30 () {56 vars in 1186 bytes} [Tue Sep 6 04:18:40 2022] GET /identity/v3/projects/1ac1bf587f2b4700b21dd5043c4bd7c2 => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:18:40.657477 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375af321-324c-453d-b9cf-026d29d94522 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:18:40.756396 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375af321-324c-453d-b9cf-026d29d94522 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:18:40.790257 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac93e563-ff13-4a23-8c39-a7e7085b211d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:18:40.790906 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac93e563-ff13-4a23-8c39-a7e7085b211d None admin] REQUEST_METHOD: `PUT` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:18:40.791180 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac93e563-ff13-4a23-8c39-a7e7085b211d None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:18:40.791448 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac93e563-ff13-4a23-8c39-a7e7085b211d None admin] PATH_INFO: `/v3/projects/1ac1bf587f2b4700b21dd5043c4bd7c2/users/8768ff5946b14186a24648064c3975ec/roles/ea4c814713b149108e488de855b025e4` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:18:40.793352 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac93e563-ff13-4a23-8c39-a7e7085b211d None admin] RBAC: Authorizing `identity:create_grant(project_id=1ac1bf587f2b4700b21dd5043c4bd7c2, user_id=8768ff5946b14186a24648064c3975ec, role_id=ea4c814713b149108e488de855b025e4)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:18:40.794753 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac93e563-ff13-4a23-8c39-a7e7085b211d None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:18:40.861117 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 550/1084] 10.222.0.30 () {58 vars in 1439 bytes} [Tue Sep 6 04:18:40 2022] PUT /identity/v3/projects/1ac1bf587f2b4700b21dd5043c4bd7c2/users/8768ff5946b14186a24648064c3975ec/roles/ea4c814713b149108e488de855b025e4 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 04:18:42.249811 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee13697-0adb-40d1-9232-7fbff1253fc5 None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:18:42.250339 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee13697-0adb-40d1-9232-7fbff1253fc5 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:18:42.250452 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee13697-0adb-40d1-9232-7fbff1253fc5 None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:18:42.252133 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 535/1085] 10.222.0.30 () {54 vars in 927 bytes} [Tue Sep 6 04:18:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 04:18:42.257345 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50dd966-c71d-4eca-8e70-8de5c0a09e68 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:18:42.257750 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50dd966-c71d-4eca-8e70-8de5c0a09e68 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:18:42.258011 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50dd966-c71d-4eca-8e70-8de5c0a09e68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:18:42.276100 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-a50dd966-c71d-4eca-8e70-8de5c0a09e68 None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:18:42.277264 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-a50dd966-c71d-4eca-8e70-8de5c0a09e68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:18:42.304111 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:18:42.344369 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 551/1086] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Sep 6 04:18:42 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 04:18:42.614106 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee13697-0adb-40d1-9232-7fbff1253fc5 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:18:42.663355 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:18:42.713563 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee13697-0adb-40d1-9232-7fbff1253fc5 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:18:42.746410 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd67e67a-7ad1-4703-b1d7-94a4b55cfa79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:18:42.747037 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd67e67a-7ad1-4703-b1d7-94a4b55cfa79 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:18:42.747312 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd67e67a-7ad1-4703-b1d7-94a4b55cfa79 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:18:42.747581 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd67e67a-7ad1-4703-b1d7-94a4b55cfa79 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:18:42.758173 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd67e67a-7ad1-4703-b1d7-94a4b55cfa79 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 04:18:42.762879 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-cd67e67a-7ad1-4703-b1d7-94a4b55cfa79 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 06 04:18:42.762879 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 04:18:42.762879 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 04:18:42.762879 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 04:18:42.762879 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 04:18:42.762879 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 04:18:42.762879 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 04:18:42.762879 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 04:18:42.762879 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 04:18:42.762879 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 04:18:42.762879 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 04:18:42.762879 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 04:18:42.762879 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 04:18:42.762879 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 04:18:42.762879 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 04:18:42.762879 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 04:18:42.763963 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 04:18:42.763963 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 04:18:42.763963 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 04:18:42.763963 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 04:18:42.763963 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 04:18:42.763963 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 04:18:42.763963 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 04:18:42.763963 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 04:18:42.763963 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 04:18:42.763963 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 04:18:42.763963 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 04:18:42.763963 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 04:18:42.763963 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 04:18:42.763963 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 04:18:42.763963 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 04:18:42.763963 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 04:18:42.764843 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 04:18:42.764843 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 04:18:42.764843 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 04:18:42.764843 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 04:18:42.764843 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 04:18:42.764843 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 04:18:42.764843 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 04:18:42.764843 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 04:18:42.764843 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 04:18:42.764843 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 04:18:42.764843 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 06 04:18:42.764843 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 04:18:42.764843 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 536/1087] 10.222.0.30 () {56 vars in 1120 bytes} [Tue Sep 6 04:18:42 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 04:18:42.769407 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3fe845-e378-4ee5-955d-69e6a07e65a0 None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:18:42.769757 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3fe845-e378-4ee5-955d-69e6a07e65a0 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:18:42.769893 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd67e67a-7ad1-4703-b1d7-94a4b55cfa79 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:18:42.770151 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3fe845-e378-4ee5-955d-69e6a07e65a0 None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:18:42.771485 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 552/1088] 10.222.0.30 () {54 vars in 927 bytes} [Tue Sep 6 04:18:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 06 04:18:42.776469 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894eaba2-9103-4476-9797-7747bc0212b7 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:18:42.776763 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894eaba2-9103-4476-9797-7747bc0212b7 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:18:42.777039 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894eaba2-9103-4476-9797-7747bc0212b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:18:42.794903 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-894eaba2-9103-4476-9797-7747bc0212b7 None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:18:42.796096 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-894eaba2-9103-4476-9797-7747bc0212b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:18:42.862712 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 553/1089] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Sep 6 04:18:42 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 06 04:18:42.869616 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd67e67a-7ad1-4703-b1d7-94a4b55cfa79 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:18:42.879533 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:18:42.905126 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d8388b-6320-4a30-83b3-ce8505907fb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:18:42.905769 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d8388b-6320-4a30-83b3-ce8505907fb6 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:18:42.906046 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d8388b-6320-4a30-83b3-ce8505907fb6 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:18:42.906312 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d8388b-6320-4a30-83b3-ce8505907fb6 None admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:18:42.906967 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d8388b-6320-4a30-83b3-ce8505907fb6 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:18:42.907263 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d8388b-6320-4a30-83b3-ce8505907fb6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:18:42.908832 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d8388b-6320-4a30-83b3-ce8505907fb6 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:18:42.921002 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 537/1090] 10.222.0.30 () {56 vars in 1115 bytes} [Tue Sep 6 04:18:42 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 04:18:42.926998 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894eaba2-9103-4476-9797-7747bc0212b7 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:18:43.016644 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:18:43.025252 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894eaba2-9103-4476-9797-7747bc0212b7 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:18:43.060161 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed19cac-78da-4c82-b0c5-e38d62a0ed9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:18:43.060787 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed19cac-78da-4c82-b0c5-e38d62a0ed9e None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:18:43.061068 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed19cac-78da-4c82-b0c5-e38d62a0ed9e None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:18:43.061365 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed19cac-78da-4c82-b0c5-e38d62a0ed9e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:18:43.071454 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed19cac-78da-4c82-b0c5-e38d62a0ed9e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:18:43.072907 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed19cac-78da-4c82-b0c5-e38d62a0ed9e None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:18:43.087169 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-fed19cac-78da-4c82-b0c5-e38d62a0ed9e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 04:18:43.087169 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 04:18:43.087169 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 04:18:43.087169 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 04:18:43.087169 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 04:18:43.087169 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 04:18:43.087169 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 04:18:43.087169 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 04:18:43.087169 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 04:18:43.087169 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 04:18:43.087169 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 04:18:43.087169 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 04:18:43.087169 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 04:18:43.087169 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 04:18:43.087169 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 04:18:43.087169 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 04:18:43.088299 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 04:18:43.088299 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 04:18:43.088299 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 04:18:43.088299 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 04:18:43.088299 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 04:18:43.088299 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 04:18:43.088299 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 04:18:43.088299 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 04:18:43.088299 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 04:18:43.088299 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 04:18:43.088299 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 04:18:43.088299 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 04:18:43.088299 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 04:18:43.088299 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 04:18:43.088299 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 04:18:43.088299 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 04:18:43.088299 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 04:18:43.089502 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 04:18:43.089502 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 04:18:43.089502 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 04:18:43.093168 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 554/1091] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Sep 6 04:18:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 04:18:43.094202 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d8388b-6320-4a30-83b3-ce8505907fb6 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:18:43.142369 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed19cac-78da-4c82-b0c5-e38d62a0ed9e None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:18:43.194064 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d8388b-6320-4a30-83b3-ce8505907fb6 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:18:43.228318 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc6c01c-47af-48ec-878c-12db3a75a492 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:18:43.229186 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc6c01c-47af-48ec-878c-12db3a75a492 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:18:43.229632 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc6c01c-47af-48ec-878c-12db3a75a492 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:18:43.230035 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc6c01c-47af-48ec-878c-12db3a75a492 None admin] PATH_INFO: `/v3/domains` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:18:43.231011 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc6c01c-47af-48ec-878c-12db3a75a492 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:18:43.231496 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc6c01c-47af-48ec-878c-12db3a75a492 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:18:43.233635 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc6c01c-47af-48ec-878c-12db3a75a492 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:18:43.244220 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed19cac-78da-4c82-b0c5-e38d62a0ed9e None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:18:43.251927 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 538/1092] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Sep 6 04:18:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:18:43.257233 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc6c01c-47af-48ec-878c-12db3a75a492 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:18:43.283573 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6522499-b038-4d37-ad1d-4cc86f9694c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:18:43.284157 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6522499-b038-4d37-ad1d-4cc86f9694c4 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:18:43.284344 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6522499-b038-4d37-ad1d-4cc86f9694c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:18:43.284622 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6522499-b038-4d37-ad1d-4cc86f9694c4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:18:43.293586 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:18:43.295404 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6522499-b038-4d37-ad1d-4cc86f9694c4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 04:18:43.300953 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-a6522499-b038-4d37-ad1d-4cc86f9694c4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 04:18:43.300953 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 04:18:43.300953 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 04:18:43.300953 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 04:18:43.300953 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 04:18:43.300953 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 04:18:43.300953 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 04:18:43.300953 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 04:18:43.300953 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 04:18:43.300953 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 04:18:43.300953 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 04:18:43.300953 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 04:18:43.300953 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 04:18:43.300953 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 04:18:43.300953 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 04:18:43.300953 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 04:18:43.302062 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 04:18:43.302062 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 04:18:43.302062 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 04:18:43.302062 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 04:18:43.302062 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 04:18:43.302062 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 04:18:43.302062 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 04:18:43.302062 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 04:18:43.302062 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 04:18:43.302062 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 04:18:43.302062 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 04:18:43.302062 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 04:18:43.302062 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 04:18:43.302062 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 04:18:43.302062 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 04:18:43.302062 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 04:18:43.302924 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 04:18:43.302924 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 04:18:43.302924 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 04:18:43.302924 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 04:18:43.302924 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 04:18:43.302924 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 04:18:43.302924 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 04:18:43.302924 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 04:18:43.302924 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 04:18:43.302924 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 04:18:43.302924 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 04:18:43.302924 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 04:18:43.302924 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 555/1093] 10.222.0.30 () {56 vars in 1096 bytes} [Tue Sep 6 04:18:43 2022] GET /identity/v3/roles/admin => generated 83 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 04:18:43.308279 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6522499-b038-4d37-ad1d-4cc86f9694c4 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:18:43.312397 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:18:43.373815 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc6c01c-47af-48ec-878c-12db3a75a492 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:18:43.411470 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c31759-628f-4ef1-8fc0-157287c62173 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:18:43.412043 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c31759-628f-4ef1-8fc0-157287c62173 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:18:43.412324 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c31759-628f-4ef1-8fc0-157287c62173 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:18:43.412594 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c31759-628f-4ef1-8fc0-157287c62173 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:18:43.418579 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6522499-b038-4d37-ad1d-4cc86f9694c4 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:18:43.419703 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c31759-628f-4ef1-8fc0-157287c62173 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:18:43.421133 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c31759-628f-4ef1-8fc0-157287c62173 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:18:43.433142 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-a6c31759-628f-4ef1-8fc0-157287c62173 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 06 04:18:43.433142 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 04:18:43.433142 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 04:18:43.433142 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 04:18:43.433142 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 04:18:43.433142 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 04:18:43.433142 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 04:18:43.433142 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 04:18:43.433142 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 04:18:43.433142 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 04:18:43.433142 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 04:18:43.433142 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 04:18:43.433142 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 06 04:18:43.433142 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 06 04:18:43.433142 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 06 04:18:43.433142 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 06 04:18:43.434199 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 04:18:43.434199 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 04:18:43.434199 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 06 04:18:43.434199 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 04:18:43.434199 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 06 04:18:43.434199 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 04:18:43.434199 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 04:18:43.434199 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 04:18:43.434199 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 04:18:43.434199 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 04:18:43.434199 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 04:18:43.434199 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 04:18:43.434199 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 04:18:43.434199 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 04:18:43.434199 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 04:18:43.434199 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 04:18:43.434199 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 04:18:43.434199 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 04:18:43.440464 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 04:18:43.440464 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 04:18:43.440464 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 06 04:18:43.440464 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 06 04:18:43.440464 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 06 04:18:43.440464 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 06 04:18:43.440464 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 06 04:18:43.440464 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 06 04:18:43.440464 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 06 04:18:43.440464 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 04:18:43.440464 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 539/1094] 10.222.0.30 () {56 vars in 1093 bytes} [Tue Sep 6 04:18:43 2022] GET /identity/v3/users/nova => generated 82 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 04:18:43.441083 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c31759-628f-4ef1-8fc0-157287c62173 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:18:43.452298 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b292105a-f245-4c7d-a030-40853600dcdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:18:43.452902 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b292105a-f245-4c7d-a030-40853600dcdd None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:18:43.456432 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b292105a-f245-4c7d-a030-40853600dcdd None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:18:43.456432 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b292105a-f245-4c7d-a030-40853600dcdd None admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:18:43.456432 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b292105a-f245-4c7d-a030-40853600dcdd None admin] RBAC: Adding query filter params (name=admin) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:18:43.456432 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b292105a-f245-4c7d-a030-40853600dcdd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:18:43.456778 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b292105a-f245-4c7d-a030-40853600dcdd None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:18:43.465542 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 556/1095] 10.222.0.30 () {56 vars in 1099 bytes} [Tue Sep 6 04:18:43 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 04:18:43.472064 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b292105a-f245-4c7d-a030-40853600dcdd None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:18:43.534429 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:18:43.539318 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:18:43.543715 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c31759-628f-4ef1-8fc0-157287c62173 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:18:43.574216 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b292105a-f245-4c7d-a030-40853600dcdd None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:18:43.585039 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846472ff-ae90-4fa7-93c9-aa0587eeaf09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:18:43.585800 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846472ff-ae90-4fa7-93c9-aa0587eeaf09 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:18:43.586085 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846472ff-ae90-4fa7-93c9-aa0587eeaf09 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:18:43.586352 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846472ff-ae90-4fa7-93c9-aa0587eeaf09 None admin] PATH_INFO: `/v3/users` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:18:43.587091 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846472ff-ae90-4fa7-93c9-aa0587eeaf09 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:18:43.587464 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846472ff-ae90-4fa7-93c9-aa0587eeaf09 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:18:43.588816 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846472ff-ae90-4fa7-93c9-aa0587eeaf09 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:18:43.606815 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 540/1096] 10.222.0.30 () {56 vars in 1133 bytes} [Tue Sep 6 04:18:43 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:18:43.611462 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722b2cce-7627-4185-b163-4cb42d148fea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:18:43.612226 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722b2cce-7627-4185-b163-4cb42d148fea None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:18:43.612508 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722b2cce-7627-4185-b163-4cb42d148fea None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:18:43.612778 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722b2cce-7627-4185-b163-4cb42d148fea None admin] PATH_INFO: `/v3/users/8768ff5946b14186a24648064c3975ec` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:18:43.612880 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846472ff-ae90-4fa7-93c9-aa0587eeaf09 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:18:43.613923 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722b2cce-7627-4185-b163-4cb42d148fea None admin] RBAC: Authorizing `identity:get_user(user_id=8768ff5946b14186a24648064c3975ec)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:18:43.615357 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722b2cce-7627-4185-b163-4cb42d148fea None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:18:43.621796 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 557/1097] 10.222.0.30 () {56 vars in 1177 bytes} [Tue Sep 6 04:18:43 2022] GET /identity/v3/users/8768ff5946b14186a24648064c3975ec => generated 285 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 04:18:43.622573 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722b2cce-7627-4185-b163-4cb42d148fea None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:18:43.711491 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846472ff-ae90-4fa7-93c9-aa0587eeaf09 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:18:43.722133 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722b2cce-7627-4185-b163-4cb42d148fea None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:18:43.744610 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1516c02-e13e-4f54-a767-5fa7b40b3b78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:18:43.745216 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1516c02-e13e-4f54-a767-5fa7b40b3b78 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:18:43.745524 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1516c02-e13e-4f54-a767-5fa7b40b3b78 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:18:43.745800 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1516c02-e13e-4f54-a767-5fa7b40b3b78 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:18:43.756797 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8bca851-a677-4683-8d8c-ddc66b5f697b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:18:43.757427 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bca851-a677-4683-8d8c-ddc66b5f697b None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:18:43.757706 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bca851-a677-4683-8d8c-ddc66b5f697b None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:18:43.757974 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bca851-a677-4683-8d8c-ddc66b5f697b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:18:43.760472 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:18:43.763621 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:18:43.812148 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1516c02-e13e-4f54-a767-5fa7b40b3b78 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:18:43.813680 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1516c02-e13e-4f54-a767-5fa7b40b3b78 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:18:43.826758 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8bca851-a677-4683-8d8c-ddc66b5f697b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:18:43.828193 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8bca851-a677-4683-8d8c-ddc66b5f697b None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:18:43.914197 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 541/1098] 10.222.0.30 () {58 vars in 1282 bytes} [Tue Sep 6 04:18:43 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 06 04:18:43.919886 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1516c02-e13e-4f54-a767-5fa7b40b3b78 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:18:43.934634 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 558/1099] 10.222.0.30 () {58 vars in 1282 bytes} [Tue Sep 6 04:18:43 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 06 04:18:43.935655 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8bca851-a677-4683-8d8c-ddc66b5f697b None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:18:43.966635 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:18:43.967671 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:18:44.017663 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1516c02-e13e-4f54-a767-5fa7b40b3b78 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:18:44.032786 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8bca851-a677-4683-8d8c-ddc66b5f697b None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:18:44.055316 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5120c2a-7cac-46e6-8620-60dd2bbc253a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:18:44.060345 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5120c2a-7cac-46e6-8620-60dd2bbc253a None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:18:44.060666 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5120c2a-7cac-46e6-8620-60dd2bbc253a None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:18:44.060945 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5120c2a-7cac-46e6-8620-60dd2bbc253a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:18:44.067376 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3822e006-0508-498b-9303-40593fe06467 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:18:44.068103 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3822e006-0508-498b-9303-40593fe06467 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:18:44.068290 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3822e006-0508-498b-9303-40593fe06467 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:18:44.068563 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3822e006-0508-498b-9303-40593fe06467 None admin] PATH_INFO: `/v3/projects/1ac1bf587f2b4700b21dd5043c4bd7c2` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:18:44.069726 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3822e006-0508-498b-9303-40593fe06467 None admin] RBAC: Authorizing `identity:get_project(project_id=1ac1bf587f2b4700b21dd5043c4bd7c2)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:18:44.071169 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3822e006-0508-498b-9303-40593fe06467 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:18:44.072386 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5120c2a-7cac-46e6-8620-60dd2bbc253a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:18:44.072812 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 559/1100] 10.222.0.30 () {56 vars in 1186 bytes} [Tue Sep 6 04:18:43 2022] GET /identity/v3/projects/1ac1bf587f2b4700b21dd5043c4bd7c2 => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 04:18:44.077914 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5120c2a-7cac-46e6-8620-60dd2bbc253a None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:18:44.078695 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3822e006-0508-498b-9303-40593fe06467 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:18:44.088966 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-a5120c2a-7cac-46e6-8620-60dd2bbc253a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 04:18:44.088966 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 04:18:44.088966 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 04:18:44.088966 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 04:18:44.088966 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 04:18:44.088966 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 04:18:44.088966 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 04:18:44.088966 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 04:18:44.088966 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 04:18:44.088966 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 04:18:44.088966 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 04:18:44.088966 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 04:18:44.088966 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 04:18:44.088966 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 04:18:44.088966 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 04:18:44.088966 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 04:18:44.090165 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 04:18:44.090165 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 04:18:44.090165 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 04:18:44.090165 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 04:18:44.090165 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 04:18:44.090165 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 04:18:44.090165 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 04:18:44.090165 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 04:18:44.090165 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 04:18:44.090165 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 04:18:44.090165 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 04:18:44.090165 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 04:18:44.090165 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 04:18:44.090165 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 04:18:44.090165 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 04:18:44.090165 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 04:18:44.090165 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 04:18:44.091251 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 04:18:44.091251 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 04:18:44.091251 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 04:18:44.091251 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 542/1101] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Sep 6 04:18:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 04:18:44.096500 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5120c2a-7cac-46e6-8620-60dd2bbc253a None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:18:44.182218 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3822e006-0508-498b-9303-40593fe06467 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:18:44.183690 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:18:44.195987 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:18:44.198898 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5120c2a-7cac-46e6-8620-60dd2bbc253a None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:18:44.216065 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820df4ef-dbe8-4e42-bc30-21ef3c9dc3a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:18:44.216716 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820df4ef-dbe8-4e42-bc30-21ef3c9dc3a0 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:18:44.216939 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820df4ef-dbe8-4e42-bc30-21ef3c9dc3a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:18:44.217205 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820df4ef-dbe8-4e42-bc30-21ef3c9dc3a0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:18:44.217953 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-820df4ef-dbe8-4e42-bc30-21ef3c9dc3a0 None admin] RBAC: Adding query filter params (role.id=ea4c814713b149108e488de855b025e4, scope.project.id=1ac1bf587f2b4700b21dd5043c4bd7c2, user.id=8768ff5946b14186a24648064c3975ec) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:18:44.218254 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-820df4ef-dbe8-4e42-bc30-21ef3c9dc3a0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:18:44.219684 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-820df4ef-dbe8-4e42-bc30-21ef3c9dc3a0 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:18:44.227269 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 560/1102] 10.222.0.30 () {56 vars in 1374 bytes} [Tue Sep 6 04:18:44 2022] GET /identity/v3/role_assignments?user.id=8768ff5946b14186a24648064c3975ec&scope.project.id=1ac1bf587f2b4700b21dd5043c4bd7c2&role.id=ea4c814713b149108e488de855b025e4 => generated 608 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 04:18:44.232560 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff940b22-f825-4ffc-8933-e1e493329dc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:18:44.233185 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff940b22-f825-4ffc-8933-e1e493329dc5 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:18:44.233498 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff940b22-f825-4ffc-8933-e1e493329dc5 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:18:44.233766 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff940b22-f825-4ffc-8933-e1e493329dc5 None admin] PATH_INFO: `/v3/domains` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:18:44.234413 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff940b22-f825-4ffc-8933-e1e493329dc5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:18:44.234712 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff940b22-f825-4ffc-8933-e1e493329dc5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:18:44.236408 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff940b22-f825-4ffc-8933-e1e493329dc5 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:18:44.250574 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 543/1103] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Sep 6 04:18:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 04:18:44.256775 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820df4ef-dbe8-4e42-bc30-21ef3c9dc3a0 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:18:44.362793 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820df4ef-dbe8-4e42-bc30-21ef3c9dc3a0 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:18:44.399968 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55108b2c-1685-4c6e-8e24-a3b2413b5d5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:18:44.400601 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55108b2c-1685-4c6e-8e24-a3b2413b5d5a None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:18:44.400876 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55108b2c-1685-4c6e-8e24-a3b2413b5d5a None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:18:44.401143 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55108b2c-1685-4c6e-8e24-a3b2413b5d5a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:18:44.408495 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55108b2c-1685-4c6e-8e24-a3b2413b5d5a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:18:44.409982 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55108b2c-1685-4c6e-8e24-a3b2413b5d5a None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:18:44.421262 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-55108b2c-1685-4c6e-8e24-a3b2413b5d5a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 04:18:44.421262 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 04:18:44.421262 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 04:18:44.421262 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 04:18:44.421262 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 04:18:44.421262 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 04:18:44.421262 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 04:18:44.421262 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 04:18:44.421262 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 04:18:44.421262 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 04:18:44.421262 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 04:18:44.421262 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 04:18:44.421262 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 06 04:18:44.421262 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 06 04:18:44.421262 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 06 04:18:44.425140 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 06 04:18:44.425140 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 04:18:44.425140 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 04:18:44.425140 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 04:18:44.425140 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 04:18:44.425140 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 04:18:44.425140 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 04:18:44.425140 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 04:18:44.425140 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 04:18:44.425140 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 04:18:44.425140 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 04:18:44.425140 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 04:18:44.425140 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 04:18:44.425140 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 04:18:44.425140 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 04:18:44.425140 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 04:18:44.425140 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 04:18:44.426308 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 06 04:18:44.426308 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 06 04:18:44.426308 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 06 04:18:44.426308 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 06 04:18:44.426308 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 06 04:18:44.426308 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 06 04:18:44.426308 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 04:18:44.426308 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 04:18:44.426308 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 561/1104] 10.222.0.30 () {56 vars in 1111 bytes} [Tue Sep 6 04:18:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 04:18:44.431582 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff940b22-f825-4ffc-8933-e1e493329dc5 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:18:44.547832 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff940b22-f825-4ffc-8933-e1e493329dc5 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:18:44.581156 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7507be-d6d8-4766-a589-1662b946148a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:18:44.581791 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7507be-d6d8-4766-a589-1662b946148a None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:18:44.582069 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7507be-d6d8-4766-a589-1662b946148a None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:18:44.582335 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7507be-d6d8-4766-a589-1662b946148a None admin] PATH_INFO: `/v3/projects` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:18:44.583018 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7507be-d6d8-4766-a589-1662b946148a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:18:44.583318 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7507be-d6d8-4766-a589-1662b946148a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:18:44.584726 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7507be-d6d8-4766-a589-1662b946148a None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:18:44.596598 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 544/1105] 10.222.0.30 () {56 vars in 1148 bytes} [Tue Sep 6 04:18:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:18:44.602710 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55108b2c-1685-4c6e-8e24-a3b2413b5d5a None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:18:44.631275 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:18:44.706998 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55108b2c-1685-4c6e-8e24-a3b2413b5d5a None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:18:44.740659 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d64466-cd2a-4784-878e-9753b07d4e10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:18:44.741263 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d64466-cd2a-4784-878e-9753b07d4e10 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:18:44.741576 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d64466-cd2a-4784-878e-9753b07d4e10 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:18:44.741844 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d64466-cd2a-4784-878e-9753b07d4e10 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:18:44.742562 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d64466-cd2a-4784-878e-9753b07d4e10 None admin] RBAC: Adding query filter params (role.id=356631f6a0064bc695b78e3759d55d6a, scope.project.id=b707733ce65c42edbc1fd95ad0a1c06c, user.id=9db138b77afa4b068244b8ef715404fe) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:18:44.742865 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d64466-cd2a-4784-878e-9753b07d4e10 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:18:44.744283 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d64466-cd2a-4784-878e-9753b07d4e10 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:18:44.749900 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 562/1106] 10.222.0.30 () {56 vars in 1374 bytes} [Tue Sep 6 04:18:44 2022] GET /identity/v3/role_assignments?user.id=9db138b77afa4b068244b8ef715404fe&scope.project.id=b707733ce65c42edbc1fd95ad0a1c06c&role.id=356631f6a0064bc695b78e3759d55d6a => generated 608 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:18:46.161051 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fdcf2a-63bc-434f-9c9a-0e8edcde4545 None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:18:46.161616 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fdcf2a-63bc-434f-9c9a-0e8edcde4545 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:18:46.161734 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fdcf2a-63bc-434f-9c9a-0e8edcde4545 None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:18:46.163013 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 545/1107] 10.222.0.30 () {54 vars in 927 bytes} [Tue Sep 6 04:18:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 04:18:46.168252 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710126bb-3725-4e7d-a451-d6ab5a0e1861 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:18:46.168643 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710126bb-3725-4e7d-a451-d6ab5a0e1861 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:18:46.168793 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710126bb-3725-4e7d-a451-d6ab5a0e1861 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:18:46.187224 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-710126bb-3725-4e7d-a451-d6ab5a0e1861 None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:18:46.188385 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-710126bb-3725-4e7d-a451-d6ab5a0e1861 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:18:46.252912 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 563/1108] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Sep 6 04:18:46 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 04:18:46.523270 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1fdcf2a-63bc-434f-9c9a-0e8edcde4545 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:18:46.544999 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:18:46.621848 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1fdcf2a-63bc-434f-9c9a-0e8edcde4545 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:18:46.654171 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8db9218-b25c-4834-91ce-f37acd482a2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:18:46.654787 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8db9218-b25c-4834-91ce-f37acd482a2f None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:18:46.655061 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8db9218-b25c-4834-91ce-f37acd482a2f None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:18:46.655328 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8db9218-b25c-4834-91ce-f37acd482a2f None admin] PATH_INFO: `/v3/domains/default` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:18:46.656433 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8db9218-b25c-4834-91ce-f37acd482a2f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:18:46.657913 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8db9218-b25c-4834-91ce-f37acd482a2f None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:18:46.659446 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 546/1109] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Sep 6 04:18:46 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:18:46.665013 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710126bb-3725-4e7d-a451-d6ab5a0e1861 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:18:46.685698 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:18:46.760724 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710126bb-3725-4e7d-a451-d6ab5a0e1861 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:18:46.793513 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49e70e3-e290-41df-97ea-d574598765b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:18:46.794052 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49e70e3-e290-41df-97ea-d574598765b9 None admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:18:46.794327 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49e70e3-e290-41df-97ea-d574598765b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:18:46.794595 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49e70e3-e290-41df-97ea-d574598765b9 None admin] PATH_INFO: `/v3/users` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:18:46.795245 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49e70e3-e290-41df-97ea-d574598765b9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:18:46.796661 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49e70e3-e290-41df-97ea-d574598765b9 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:18:46.902394 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 564/1110] 10.222.0.30 () {60 vars in 1132 bytes} [Tue Sep 6 04:18:46 2022] POST /identity/v3/users => generated 286 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 04:18:48.831534 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7a7504-0f74-4beb-bc12-5e4e8e5ee7bd None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:18:48.832075 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7a7504-0f74-4beb-bc12-5e4e8e5ee7bd None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:18:48.832075 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7a7504-0f74-4beb-bc12-5e4e8e5ee7bd None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:18:48.833450 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 547/1111] 10.222.0.30 () {54 vars in 927 bytes} [Tue Sep 6 04:18:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 04:18:48.838456 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a0ecf9-1a4b-4da1-a18e-7ca6d196c749 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:18:48.838830 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a0ecf9-1a4b-4da1-a18e-7ca6d196c749 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:18:48.838994 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a0ecf9-1a4b-4da1-a18e-7ca6d196c749 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:18:48.857235 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-81a0ecf9-1a4b-4da1-a18e-7ca6d196c749 None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:18:48.858427 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-81a0ecf9-1a4b-4da1-a18e-7ca6d196c749 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:18:48.923569 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 565/1112] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Sep 6 04:18:48 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 06 04:18:49.201068 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7a7504-0f74-4beb-bc12-5e4e8e5ee7bd None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:18:49.288124 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:18:49.301787 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7a7504-0f74-4beb-bc12-5e4e8e5ee7bd None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:18:49.335087 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26985f27-96eb-4448-b364-2f275b72c924 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:18:49.335711 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26985f27-96eb-4448-b364-2f275b72c924 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:18:49.335991 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26985f27-96eb-4448-b364-2f275b72c924 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:18:49.336264 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26985f27-96eb-4448-b364-2f275b72c924 None admin] PATH_INFO: `/v3/roles/77197446dd2541b9a4fab9a654c08fc9` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:18:49.337669 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26985f27-96eb-4448-b364-2f275b72c924 None admin] RBAC: Authorizing `identity:get_role(role_id=77197446dd2541b9a4fab9a654c08fc9)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:18:49.339026 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26985f27-96eb-4448-b364-2f275b72c924 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:18:49.340332 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 548/1113] 10.222.0.30 () {56 vars in 1177 bytes} [Tue Sep 6 04:18:49 2022] GET /identity/v3/roles/77197446dd2541b9a4fab9a654c08fc9 => generated 223 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:18:49.347093 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a0ecf9-1a4b-4da1-a18e-7ca6d196c749 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:18:49.356007 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:18:49.447443 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a0ecf9-1a4b-4da1-a18e-7ca6d196c749 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:18:49.482289 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1db05fa-e0f0-46bd-a2a2-a005a965dd11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:18:49.482860 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1db05fa-e0f0-46bd-a2a2-a005a965dd11 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:18:49.483140 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1db05fa-e0f0-46bd-a2a2-a005a965dd11 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:18:49.483415 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1db05fa-e0f0-46bd-a2a2-a005a965dd11 None admin] PATH_INFO: `/v3/users/e8fefbbcaa3f49e88421551f3dcc5a53` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:18:49.496137 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1db05fa-e0f0-46bd-a2a2-a005a965dd11 None admin] RBAC: Authorizing `identity:get_user(user_id=e8fefbbcaa3f49e88421551f3dcc5a53)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:18:49.497609 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1db05fa-e0f0-46bd-a2a2-a005a965dd11 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:18:49.499244 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 566/1114] 10.222.0.30 () {56 vars in 1177 bytes} [Tue Sep 6 04:18:49 2022] GET /identity/v3/users/e8fefbbcaa3f49e88421551f3dcc5a53 => generated 286 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:18:49.505418 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26985f27-96eb-4448-b364-2f275b72c924 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:18:49.604563 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26985f27-96eb-4448-b364-2f275b72c924 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:18:49.637751 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3c0a88-e861-4a58-bd24-efe16c058d6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:18:49.638356 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3c0a88-e861-4a58-bd24-efe16c058d6f None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:18:49.638638 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3c0a88-e861-4a58-bd24-efe16c058d6f None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:18:49.638913 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3c0a88-e861-4a58-bd24-efe16c058d6f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:18:49.663755 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:18:49.705957 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3c0a88-e861-4a58-bd24-efe16c058d6f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:18:49.707433 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3c0a88-e861-4a58-bd24-efe16c058d6f None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:18:49.803172 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 549/1115] 10.222.0.30 () {58 vars in 1282 bytes} [Tue Sep 6 04:18:49 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 06 04:18:49.809171 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1db05fa-e0f0-46bd-a2a2-a005a965dd11 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:18:49.870150 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:18:49.906884 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1db05fa-e0f0-46bd-a2a2-a005a965dd11 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:18:49.939248 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871ca192-6764-4240-8b5b-8d6d51bf5976 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:18:49.939867 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871ca192-6764-4240-8b5b-8d6d51bf5976 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:18:49.940139 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871ca192-6764-4240-8b5b-8d6d51bf5976 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:18:49.940405 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871ca192-6764-4240-8b5b-8d6d51bf5976 None admin] PATH_INFO: `/v3/projects/1ac1bf587f2b4700b21dd5043c4bd7c2` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:18:49.941546 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-871ca192-6764-4240-8b5b-8d6d51bf5976 None admin] RBAC: Authorizing `identity:get_project(project_id=1ac1bf587f2b4700b21dd5043c4bd7c2)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:18:49.942969 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-871ca192-6764-4240-8b5b-8d6d51bf5976 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:18:49.944675 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 567/1116] 10.222.0.30 () {56 vars in 1186 bytes} [Tue Sep 6 04:18:49 2022] GET /identity/v3/projects/1ac1bf587f2b4700b21dd5043c4bd7c2 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:18:49.950857 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3c0a88-e861-4a58-bd24-efe16c058d6f None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:18:50.006294 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:18:50.048708 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3c0a88-e861-4a58-bd24-efe16c058d6f None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:18:50.081701 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01f27fd-6f31-4fe0-abd3-4b07e241b085 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:18:50.082240 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01f27fd-6f31-4fe0-abd3-4b07e241b085 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:18:50.082523 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01f27fd-6f31-4fe0-abd3-4b07e241b085 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:18:50.082797 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01f27fd-6f31-4fe0-abd3-4b07e241b085 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:18:50.083517 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01f27fd-6f31-4fe0-abd3-4b07e241b085 None admin] RBAC: Adding query filter params (role.id=77197446dd2541b9a4fab9a654c08fc9, scope.project.id=1ac1bf587f2b4700b21dd5043c4bd7c2, user.id=e8fefbbcaa3f49e88421551f3dcc5a53) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:18:50.083824 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01f27fd-6f31-4fe0-abd3-4b07e241b085 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:18:50.085238 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01f27fd-6f31-4fe0-abd3-4b07e241b085 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:18:50.090497 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 550/1117] 10.222.0.30 () {56 vars in 1374 bytes} [Tue Sep 6 04:18:49 2022] GET /identity/v3/role_assignments?user.id=e8fefbbcaa3f49e88421551f3dcc5a53&scope.project.id=1ac1bf587f2b4700b21dd5043c4bd7c2&role.id=77197446dd2541b9a4fab9a654c08fc9 => generated 259 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:18:51.994162 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d16022-d754-48a6-a18c-0afc65be8d1c None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:18:51.994683 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d16022-d754-48a6-a18c-0afc65be8d1c None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:18:51.994773 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d16022-d754-48a6-a18c-0afc65be8d1c None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:18:51.996049 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 568/1118] 10.222.0.30 () {54 vars in 927 bytes} [Tue Sep 6 04:18:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 04:18:52.002336 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebedfac1-3e82-4a21-904f-ceacf46ec18f None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:18:52.002750 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebedfac1-3e82-4a21-904f-ceacf46ec18f None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:18:52.002890 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebedfac1-3e82-4a21-904f-ceacf46ec18f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:18:52.021557 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-ebedfac1-3e82-4a21-904f-ceacf46ec18f None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:18:52.022747 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-ebedfac1-3e82-4a21-904f-ceacf46ec18f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:18:52.087324 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 551/1119] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Sep 6 04:18:51 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 04:18:52.368374 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d16022-d754-48a6-a18c-0afc65be8d1c None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:18:52.469376 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d16022-d754-48a6-a18c-0afc65be8d1c None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:18:52.497621 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:18:52.503334 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76288142-4d44-425a-b150-94fb9e0f1762 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:18:52.503957 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76288142-4d44-425a-b150-94fb9e0f1762 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:18:52.504232 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76288142-4d44-425a-b150-94fb9e0f1762 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:18:52.504497 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76288142-4d44-425a-b150-94fb9e0f1762 None admin] PATH_INFO: `/v3/roles/77197446dd2541b9a4fab9a654c08fc9` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:18:52.505970 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76288142-4d44-425a-b150-94fb9e0f1762 None admin] RBAC: Authorizing `identity:get_role(role_id=77197446dd2541b9a4fab9a654c08fc9)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:18:52.507324 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76288142-4d44-425a-b150-94fb9e0f1762 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:18:52.521360 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 569/1120] 10.222.0.30 () {56 vars in 1177 bytes} [Tue Sep 6 04:18:52 2022] GET /identity/v3/roles/77197446dd2541b9a4fab9a654c08fc9 => generated 223 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:18:52.521490 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebedfac1-3e82-4a21-904f-ceacf46ec18f None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:18:52.571800 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:18:52.620104 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebedfac1-3e82-4a21-904f-ceacf46ec18f None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:18:52.653000 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67076716-42a0-4f39-8a23-fbd1f50c45e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:18:52.653666 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67076716-42a0-4f39-8a23-fbd1f50c45e9 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:18:52.653944 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67076716-42a0-4f39-8a23-fbd1f50c45e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:18:52.654215 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67076716-42a0-4f39-8a23-fbd1f50c45e9 None admin] PATH_INFO: `/v3/users/e8fefbbcaa3f49e88421551f3dcc5a53` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:18:52.655311 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67076716-42a0-4f39-8a23-fbd1f50c45e9 None admin] RBAC: Authorizing `identity:get_user(user_id=e8fefbbcaa3f49e88421551f3dcc5a53)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:18:52.656723 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67076716-42a0-4f39-8a23-fbd1f50c45e9 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:18:52.658348 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 552/1121] 10.222.0.30 () {56 vars in 1177 bytes} [Tue Sep 6 04:18:52 2022] GET /identity/v3/users/e8fefbbcaa3f49e88421551f3dcc5a53 => generated 286 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:18:52.664341 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76288142-4d44-425a-b150-94fb9e0f1762 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:18:52.762210 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76288142-4d44-425a-b150-94fb9e0f1762 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:18:52.795145 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec75e55-77d6-4a2d-a3c3-0d838d72f6f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:18:52.795767 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec75e55-77d6-4a2d-a3c3-0d838d72f6f2 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:18:52.796049 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec75e55-77d6-4a2d-a3c3-0d838d72f6f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:18:52.796322 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec75e55-77d6-4a2d-a3c3-0d838d72f6f2 None admin] PATH_INFO: `/v3/projects/1ac1bf587f2b4700b21dd5043c4bd7c2` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:18:52.797457 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec75e55-77d6-4a2d-a3c3-0d838d72f6f2 None admin] RBAC: Authorizing `identity:get_project(project_id=1ac1bf587f2b4700b21dd5043c4bd7c2)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:18:52.798926 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec75e55-77d6-4a2d-a3c3-0d838d72f6f2 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:18:52.800538 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 570/1122] 10.222.0.30 () {56 vars in 1186 bytes} [Tue Sep 6 04:18:52 2022] GET /identity/v3/projects/1ac1bf587f2b4700b21dd5043c4bd7c2 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:18:52.806505 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67076716-42a0-4f39-8a23-fbd1f50c45e9 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:18:52.906844 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67076716-42a0-4f39-8a23-fbd1f50c45e9 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:18:52.926324 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:18:52.943401 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ebebf8-ecbd-4a33-9aa8-3240726705e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:18:52.944103 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ebebf8-ecbd-4a33-9aa8-3240726705e1 None admin] REQUEST_METHOD: `PUT` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:18:52.944383 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ebebf8-ecbd-4a33-9aa8-3240726705e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:18:52.944652 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ebebf8-ecbd-4a33-9aa8-3240726705e1 None admin] PATH_INFO: `/v3/projects/1ac1bf587f2b4700b21dd5043c4bd7c2/users/e8fefbbcaa3f49e88421551f3dcc5a53/roles/77197446dd2541b9a4fab9a654c08fc9` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:18:52.946858 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ebebf8-ecbd-4a33-9aa8-3240726705e1 None admin] RBAC: Authorizing `identity:create_grant(project_id=1ac1bf587f2b4700b21dd5043c4bd7c2, user_id=e8fefbbcaa3f49e88421551f3dcc5a53, role_id=77197446dd2541b9a4fab9a654c08fc9)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:18:52.948278 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ebebf8-ecbd-4a33-9aa8-3240726705e1 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:18:53.020021 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 553/1123] 10.222.0.30 () {58 vars in 1439 bytes} [Tue Sep 6 04:18:52 2022] PUT /identity/v3/projects/1ac1bf587f2b4700b21dd5043c4bd7c2/users/e8fefbbcaa3f49e88421551f3dcc5a53/roles/77197446dd2541b9a4fab9a654c08fc9 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 04:18:54.934852 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc46824f-fd44-4dd8-8143-78724f146b8a None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:18:54.935498 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc46824f-fd44-4dd8-8143-78724f146b8a None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:18:54.935498 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc46824f-fd44-4dd8-8143-78724f146b8a None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:18:54.936750 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 571/1124] 10.222.0.30 () {54 vars in 927 bytes} [Tue Sep 6 04:18:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 04:18:54.942041 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54277bcc-07b9-469a-9761-820600ebb64f None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:18:54.942455 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54277bcc-07b9-469a-9761-820600ebb64f None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:18:54.942592 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54277bcc-07b9-469a-9761-820600ebb64f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:18:54.960385 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-54277bcc-07b9-469a-9761-820600ebb64f None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:18:54.961606 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-54277bcc-07b9-469a-9761-820600ebb64f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:18:55.026163 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 554/1125] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Sep 6 04:18:54 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 04:18:55.291562 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc46824f-fd44-4dd8-8143-78724f146b8a None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:18:55.358075 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:18:55.393027 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc46824f-fd44-4dd8-8143-78724f146b8a None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:18:55.433247 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196f07c9-5aa7-4550-a628-9ef359663ce5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:18:55.433900 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196f07c9-5aa7-4550-a628-9ef359663ce5 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:18:55.434177 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196f07c9-5aa7-4550-a628-9ef359663ce5 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:18:55.434447 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196f07c9-5aa7-4550-a628-9ef359663ce5 None admin] PATH_INFO: `/v3/roles/77197446dd2541b9a4fab9a654c08fc9` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:18:55.435813 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196f07c9-5aa7-4550-a628-9ef359663ce5 None admin] RBAC: Authorizing `identity:get_role(role_id=77197446dd2541b9a4fab9a654c08fc9)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:18:55.437177 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196f07c9-5aa7-4550-a628-9ef359663ce5 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:18:55.439107 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 572/1126] 10.222.0.30 () {56 vars in 1177 bytes} [Tue Sep 6 04:18:55 2022] GET /identity/v3/roles/77197446dd2541b9a4fab9a654c08fc9 => generated 223 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:18:55.444898 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54277bcc-07b9-469a-9761-820600ebb64f None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:18:55.542087 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54277bcc-07b9-469a-9761-820600ebb64f None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:18:55.551579 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:18:55.576900 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ec4f13-6dc1-4b87-8e81-6a2c188fc259 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:18:55.577572 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ec4f13-6dc1-4b87-8e81-6a2c188fc259 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:18:55.577853 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ec4f13-6dc1-4b87-8e81-6a2c188fc259 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:18:55.578124 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ec4f13-6dc1-4b87-8e81-6a2c188fc259 None admin] PATH_INFO: `/v3/users/e8fefbbcaa3f49e88421551f3dcc5a53` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:18:55.579568 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ec4f13-6dc1-4b87-8e81-6a2c188fc259 None admin] RBAC: Authorizing `identity:get_user(user_id=e8fefbbcaa3f49e88421551f3dcc5a53)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:18:55.580996 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ec4f13-6dc1-4b87-8e81-6a2c188fc259 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:18:55.582611 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 555/1127] 10.222.0.30 () {56 vars in 1177 bytes} [Tue Sep 6 04:18:55 2022] GET /identity/v3/users/e8fefbbcaa3f49e88421551f3dcc5a53 => generated 286 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:18:55.588599 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196f07c9-5aa7-4550-a628-9ef359663ce5 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:18:55.686548 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196f07c9-5aa7-4550-a628-9ef359663ce5 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:18:55.720815 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f427e19-f830-4d4b-93d2-b52ed8024a9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:18:55.721444 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f427e19-f830-4d4b-93d2-b52ed8024a9f None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:18:55.721726 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f427e19-f830-4d4b-93d2-b52ed8024a9f None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:18:55.722000 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f427e19-f830-4d4b-93d2-b52ed8024a9f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:18:55.725274 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:18:55.787036 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f427e19-f830-4d4b-93d2-b52ed8024a9f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:18:55.788471 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f427e19-f830-4d4b-93d2-b52ed8024a9f None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:18:55.885295 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 573/1128] 10.222.0.30 () {58 vars in 1282 bytes} [Tue Sep 6 04:18:55 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 06 04:18:55.891233 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ec4f13-6dc1-4b87-8e81-6a2c188fc259 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:18:55.987799 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ec4f13-6dc1-4b87-8e81-6a2c188fc259 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:18:56.019964 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59fb5a4c-8985-4d6a-a217-3d3636c266bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:18:56.020578 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fb5a4c-8985-4d6a-a217-3d3636c266bf None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:18:56.020857 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fb5a4c-8985-4d6a-a217-3d3636c266bf None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:18:56.021129 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fb5a4c-8985-4d6a-a217-3d3636c266bf None admin] PATH_INFO: `/v3/projects/1ac1bf587f2b4700b21dd5043c4bd7c2` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:18:56.022259 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59fb5a4c-8985-4d6a-a217-3d3636c266bf None admin] RBAC: Authorizing `identity:get_project(project_id=1ac1bf587f2b4700b21dd5043c4bd7c2)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:18:56.023676 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59fb5a4c-8985-4d6a-a217-3d3636c266bf None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:18:56.025272 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 556/1129] 10.222.0.30 () {56 vars in 1186 bytes} [Tue Sep 6 04:18:55 2022] GET /identity/v3/projects/1ac1bf587f2b4700b21dd5043c4bd7c2 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:18:56.031252 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f427e19-f830-4d4b-93d2-b52ed8024a9f None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:18:56.064845 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:18:56.126395 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f427e19-f830-4d4b-93d2-b52ed8024a9f None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:18:56.166230 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3d5999-e04e-4477-ba20-5f177815a0f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:18:56.166760 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3d5999-e04e-4477-ba20-5f177815a0f8 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:18:56.167034 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3d5999-e04e-4477-ba20-5f177815a0f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:18:56.167301 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3d5999-e04e-4477-ba20-5f177815a0f8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:18:56.168016 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3d5999-e04e-4477-ba20-5f177815a0f8 None admin] RBAC: Adding query filter params (role.id=77197446dd2541b9a4fab9a654c08fc9, scope.project.id=1ac1bf587f2b4700b21dd5043c4bd7c2, user.id=e8fefbbcaa3f49e88421551f3dcc5a53) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:18:56.168318 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3d5999-e04e-4477-ba20-5f177815a0f8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:18:56.169779 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3d5999-e04e-4477-ba20-5f177815a0f8 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:18:56.175401 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 574/1130] 10.222.0.30 () {56 vars in 1374 bytes} [Tue Sep 6 04:18:56 2022] GET /identity/v3/role_assignments?user.id=e8fefbbcaa3f49e88421551f3dcc5a53&scope.project.id=1ac1bf587f2b4700b21dd5043c4bd7c2&role.id=77197446dd2541b9a4fab9a654c08fc9 => generated 608 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:18:58.089451 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94932bc-be17-4225-a402-0c57a2eb0807 None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:18:58.090005 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94932bc-be17-4225-a402-0c57a2eb0807 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:18:58.090005 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94932bc-be17-4225-a402-0c57a2eb0807 None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:18:58.091340 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 557/1131] 10.222.0.30 () {54 vars in 927 bytes} [Tue Sep 6 04:18:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 04:18:58.096685 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1772cb19-f319-43a6-a457-08f53ce735d4 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:18:58.097107 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1772cb19-f319-43a6-a457-08f53ce735d4 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:18:58.097227 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1772cb19-f319-43a6-a457-08f53ce735d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:18:58.115090 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-1772cb19-f319-43a6-a457-08f53ce735d4 None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:18:58.116273 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-1772cb19-f319-43a6-a457-08f53ce735d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:18:58.185767 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 575/1132] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Sep 6 04:18:58 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 04:18:58.459017 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94932bc-be17-4225-a402-0c57a2eb0807 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:18:58.505169 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:18:58.558214 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94932bc-be17-4225-a402-0c57a2eb0807 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:18:58.590610 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a214364-7f53-4f85-b748-663200c129b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:18:58.591226 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a214364-7f53-4f85-b748-663200c129b7 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:18:58.591502 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a214364-7f53-4f85-b748-663200c129b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:18:58.591771 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a214364-7f53-4f85-b748-663200c129b7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:18:58.592856 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a214364-7f53-4f85-b748-663200c129b7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:18:58.594406 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a214364-7f53-4f85-b748-663200c129b7 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:18:58.595985 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 558/1133] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Sep 6 04:18:58 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:18:58.602293 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1772cb19-f319-43a6-a457-08f53ce735d4 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:18:58.636598 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:18:58.718784 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1772cb19-f319-43a6-a457-08f53ce735d4 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:18:58.752478 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2ef44f-edee-47f6-ac82-69cf8a154cdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:18:58.753084 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2ef44f-edee-47f6-ac82-69cf8a154cdb None admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:18:58.753387 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2ef44f-edee-47f6-ac82-69cf8a154cdb None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:18:58.753658 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2ef44f-edee-47f6-ac82-69cf8a154cdb None admin] PATH_INFO: `/v3/projects` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:18:58.754314 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2ef44f-edee-47f6-ac82-69cf8a154cdb None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:18:58.755728 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2ef44f-edee-47f6-ac82-69cf8a154cdb None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:18:58.860715 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 576/1134] 10.222.0.30 () {60 vars in 1141 bytes} [Tue Sep 6 04:18:58 2022] POST /identity/v3/projects => generated 311 bytes in 259 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 04:19:00.816346 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0619b33a-6f75-4917-a4a3-7a44d4e83f60 None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:19:00.816889 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0619b33a-6f75-4917-a4a3-7a44d4e83f60 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:19:00.816889 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0619b33a-6f75-4917-a4a3-7a44d4e83f60 None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:19:00.818259 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 559/1135] 10.222.0.30 () {54 vars in 927 bytes} [Tue Sep 6 04:19:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 04:19:00.823799 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8c22d0-13e8-49e5-9b7f-91d7c50dc1cc None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:19:00.824221 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8c22d0-13e8-49e5-9b7f-91d7c50dc1cc None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:19:00.824562 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8c22d0-13e8-49e5-9b7f-91d7c50dc1cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:19:00.843066 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-9b8c22d0-13e8-49e5-9b7f-91d7c50dc1cc None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:19:00.844320 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-9b8c22d0-13e8-49e5-9b7f-91d7c50dc1cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:19:00.910174 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 577/1136] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Sep 6 04:19:00 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 04:19:01.186283 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0619b33a-6f75-4917-a4a3-7a44d4e83f60 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:19:01.286716 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0619b33a-6f75-4917-a4a3-7a44d4e83f60 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:19:01.299675 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:19:01.320042 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2697c886-29e0-4191-872a-41e3f0879346 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:19:01.320658 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2697c886-29e0-4191-872a-41e3f0879346 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:19:01.320935 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2697c886-29e0-4191-872a-41e3f0879346 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:19:01.321202 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2697c886-29e0-4191-872a-41e3f0879346 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:19:01.322424 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2697c886-29e0-4191-872a-41e3f0879346 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:19:01.323836 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2697c886-29e0-4191-872a-41e3f0879346 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:19:01.325390 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 560/1137] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Sep 6 04:19:01 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:19:01.331361 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8c22d0-13e8-49e5-9b7f-91d7c50dc1cc None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:19:01.345161 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:19:01.432741 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8c22d0-13e8-49e5-9b7f-91d7c50dc1cc None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:19:01.466905 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242c2c09-352b-4334-a904-ae3fdba0ae18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:19:01.467450 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242c2c09-352b-4334-a904-ae3fdba0ae18 None admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:19:01.467733 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242c2c09-352b-4334-a904-ae3fdba0ae18 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:19:01.468007 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242c2c09-352b-4334-a904-ae3fdba0ae18 None admin] PATH_INFO: `/v3/users` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:19:01.468668 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-242c2c09-352b-4334-a904-ae3fdba0ae18 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:19:01.470231 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-242c2c09-352b-4334-a904-ae3fdba0ae18 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:19:01.582150 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 578/1138] 10.222.0.30 () {60 vars in 1132 bytes} [Tue Sep 6 04:19:01 2022] POST /identity/v3/users => generated 286 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 04:19:03.524361 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3967b1a-7bd5-4cc8-ad0c-ce9debc80c9d None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:19:03.524950 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3967b1a-7bd5-4cc8-ad0c-ce9debc80c9d None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:19:03.524950 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3967b1a-7bd5-4cc8-ad0c-ce9debc80c9d None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:19:03.526313 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 561/1139] 10.222.0.30 () {54 vars in 927 bytes} [Tue Sep 6 04:19:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 04:19:03.531418 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b70313-4407-4266-9a9b-07069ed7c609 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:19:03.532352 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b70313-4407-4266-9a9b-07069ed7c609 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:19:03.532573 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b70313-4407-4266-9a9b-07069ed7c609 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:19:03.550973 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-94b70313-4407-4266-9a9b-07069ed7c609 None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:19:03.552162 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-94b70313-4407-4266-9a9b-07069ed7c609 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:19:03.614320 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:19:03.619619 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 579/1140] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Sep 6 04:19:03 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 04:19:03.893629 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3967b1a-7bd5-4cc8-ad0c-ce9debc80c9d None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:19:03.994431 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3967b1a-7bd5-4cc8-ad0c-ce9debc80c9d None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:19:04.029160 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b740cfe4-7c1c-4055-963f-1c84a4a53e07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:19:04.029814 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b740cfe4-7c1c-4055-963f-1c84a4a53e07 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:19:04.030109 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b740cfe4-7c1c-4055-963f-1c84a4a53e07 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:19:04.030365 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b740cfe4-7c1c-4055-963f-1c84a4a53e07 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:19:04.040422 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b740cfe4-7c1c-4055-963f-1c84a4a53e07 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 04:19:04.045184 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-b740cfe4-7c1c-4055-963f-1c84a4a53e07 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 04:19:04.045184 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 04:19:04.045184 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 04:19:04.045184 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 04:19:04.045184 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 04:19:04.045184 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 04:19:04.045184 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 04:19:04.045184 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 04:19:04.045184 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 04:19:04.045184 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 04:19:04.045184 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 04:19:04.045184 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 04:19:04.045184 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 04:19:04.045184 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 04:19:04.045184 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 04:19:04.045184 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 04:19:04.046374 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 04:19:04.046374 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 04:19:04.046374 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 04:19:04.046374 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 04:19:04.046374 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 04:19:04.046374 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 04:19:04.046374 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 04:19:04.046374 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 04:19:04.046374 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 04:19:04.046374 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 04:19:04.046374 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 04:19:04.046374 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 04:19:04.046374 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 04:19:04.046374 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 04:19:04.046374 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 04:19:04.046374 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 04:19:04.048006 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 04:19:04.048006 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 04:19:04.048006 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 04:19:04.048006 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 04:19:04.048006 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 04:19:04.048006 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 04:19:04.048006 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 04:19:04.048006 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 04:19:04.048006 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 04:19:04.048006 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 04:19:04.048006 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 04:19:04.048006 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 04:19:04.048006 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 562/1141] 10.222.0.30 () {56 vars in 1096 bytes} [Tue Sep 6 04:19:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 04:19:04.052254 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b70313-4407-4266-9a9b-07069ed7c609 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:19:04.154595 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b70313-4407-4266-9a9b-07069ed7c609 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:19:04.187403 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6398b586-734d-447a-a4d1-7fdbadb362e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:19:04.188596 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6398b586-734d-447a-a4d1-7fdbadb362e1 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:19:04.188919 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6398b586-734d-447a-a4d1-7fdbadb362e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:19:04.189194 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6398b586-734d-447a-a4d1-7fdbadb362e1 None admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:19:04.189883 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6398b586-734d-447a-a4d1-7fdbadb362e1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:19:04.190184 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6398b586-734d-447a-a4d1-7fdbadb362e1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:19:04.191552 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6398b586-734d-447a-a4d1-7fdbadb362e1 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:19:04.199395 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 580/1142] 10.222.0.30 () {56 vars in 1099 bytes} [Tue Sep 6 04:19:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:19:04.205980 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b740cfe4-7c1c-4055-963f-1c84a4a53e07 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:19:04.245950 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:19:04.302660 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b740cfe4-7c1c-4055-963f-1c84a4a53e07 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:19:04.335402 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0011450f-50e4-49e1-9fb9-4192177b0411 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:19:04.336025 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0011450f-50e4-49e1-9fb9-4192177b0411 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:19:04.336300 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0011450f-50e4-49e1-9fb9-4192177b0411 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:19:04.336566 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0011450f-50e4-49e1-9fb9-4192177b0411 None admin] PATH_INFO: `/v3/users/05ef446a47544e0189480d6729f0a0c6` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:19:04.349923 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0011450f-50e4-49e1-9fb9-4192177b0411 None admin] RBAC: Authorizing `identity:get_user(user_id=05ef446a47544e0189480d6729f0a0c6)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:19:04.351335 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0011450f-50e4-49e1-9fb9-4192177b0411 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:19:04.352898 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 563/1143] 10.222.0.30 () {56 vars in 1177 bytes} [Tue Sep 6 04:19:04 2022] GET /identity/v3/users/05ef446a47544e0189480d6729f0a0c6 => generated 286 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:19:04.358680 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6398b586-734d-447a-a4d1-7fdbadb362e1 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:19:04.417107 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:19:04.457289 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6398b586-734d-447a-a4d1-7fdbadb362e1 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:19:04.490392 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1584547d-d389-4e06-b724-58e16bbe2bc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:19:04.490995 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1584547d-d389-4e06-b724-58e16bbe2bc0 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:19:04.491271 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1584547d-d389-4e06-b724-58e16bbe2bc0 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:19:04.491543 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1584547d-d389-4e06-b724-58e16bbe2bc0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:19:04.557272 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1584547d-d389-4e06-b724-58e16bbe2bc0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:19:04.558740 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1584547d-d389-4e06-b724-58e16bbe2bc0 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:19:04.614648 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:19:04.655920 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 581/1144] 10.222.0.30 () {58 vars in 1282 bytes} [Tue Sep 6 04:19:04 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 06 04:19:04.662576 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0011450f-50e4-49e1-9fb9-4192177b0411 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:19:04.756722 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:19:04.760402 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0011450f-50e4-49e1-9fb9-4192177b0411 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:19:04.795166 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ba6141-7972-4277-ade6-58fbb965c9e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:19:04.795790 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ba6141-7972-4277-ade6-58fbb965c9e9 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:19:04.796069 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ba6141-7972-4277-ade6-58fbb965c9e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:19:04.796341 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ba6141-7972-4277-ade6-58fbb965c9e9 None admin] PATH_INFO: `/v3/projects/25d5bd72251b46cd85ff59790e022398` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:19:04.800676 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ba6141-7972-4277-ade6-58fbb965c9e9 None admin] RBAC: Authorizing `identity:get_project(project_id=25d5bd72251b46cd85ff59790e022398)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:19:04.802188 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ba6141-7972-4277-ade6-58fbb965c9e9 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:19:04.803835 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 564/1145] 10.222.0.30 () {56 vars in 1186 bytes} [Tue Sep 6 04:19:04 2022] GET /identity/v3/projects/25d5bd72251b46cd85ff59790e022398 => generated 311 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:19:04.809949 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1584547d-d389-4e06-b724-58e16bbe2bc0 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:19:04.910072 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1584547d-d389-4e06-b724-58e16bbe2bc0 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:19:04.944115 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3757d379-ae93-47b5-aa64-82818cd6792e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:19:04.944714 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3757d379-ae93-47b5-aa64-82818cd6792e None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:19:04.944993 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3757d379-ae93-47b5-aa64-82818cd6792e None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:19:04.945261 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3757d379-ae93-47b5-aa64-82818cd6792e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:19:04.946008 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3757d379-ae93-47b5-aa64-82818cd6792e None admin] RBAC: Adding query filter params (role.id=ea4c814713b149108e488de855b025e4, scope.project.id=25d5bd72251b46cd85ff59790e022398, user.id=05ef446a47544e0189480d6729f0a0c6) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:19:04.946313 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3757d379-ae93-47b5-aa64-82818cd6792e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:19:04.947737 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3757d379-ae93-47b5-aa64-82818cd6792e None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:19:04.952950 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 582/1146] 10.222.0.30 () {56 vars in 1374 bytes} [Tue Sep 6 04:19:04 2022] GET /identity/v3/role_assignments?user.id=05ef446a47544e0189480d6729f0a0c6&scope.project.id=25d5bd72251b46cd85ff59790e022398&role.id=ea4c814713b149108e488de855b025e4 => generated 259 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:19:06.826289 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb146f5-0a7c-472f-b8d5-8db1723ad9c3 None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:19:06.826818 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb146f5-0a7c-472f-b8d5-8db1723ad9c3 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:19:06.826818 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb146f5-0a7c-472f-b8d5-8db1723ad9c3 None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:19:06.828170 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 565/1147] 10.222.0.30 () {54 vars in 927 bytes} [Tue Sep 6 04:19:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 04:19:06.833869 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcf944f-9359-432d-a86a-64ca6ee06800 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:19:06.834254 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcf944f-9359-432d-a86a-64ca6ee06800 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:19:06.834410 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcf944f-9359-432d-a86a-64ca6ee06800 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:19:06.852809 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-3bcf944f-9359-432d-a86a-64ca6ee06800 None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:19:06.854010 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-3bcf944f-9359-432d-a86a-64ca6ee06800 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:19:06.870627 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:19:06.918970 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 583/1148] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Sep 6 04:19:06 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 04:19:07.186461 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb146f5-0a7c-472f-b8d5-8db1723ad9c3 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:19:07.286680 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb146f5-0a7c-472f-b8d5-8db1723ad9c3 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:19:07.320130 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abf79cd-f36c-4106-ab40-6a712cc59a7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:19:07.320752 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abf79cd-f36c-4106-ab40-6a712cc59a7a None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:19:07.321031 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abf79cd-f36c-4106-ab40-6a712cc59a7a None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:19:07.321319 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abf79cd-f36c-4106-ab40-6a712cc59a7a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:19:07.331034 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9abf79cd-f36c-4106-ab40-6a712cc59a7a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 04:19:07.335763 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-9abf79cd-f36c-4106-ab40-6a712cc59a7a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 04:19:07.335763 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 04:19:07.335763 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 04:19:07.335763 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 04:19:07.335763 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 04:19:07.335763 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 04:19:07.335763 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 04:19:07.335763 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 04:19:07.335763 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 04:19:07.335763 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 04:19:07.335763 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 04:19:07.335763 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 04:19:07.335763 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 04:19:07.335763 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 04:19:07.335763 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 04:19:07.335763 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 04:19:07.336853 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 04:19:07.336853 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 04:19:07.336853 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 04:19:07.336853 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 04:19:07.336853 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 04:19:07.336853 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 04:19:07.336853 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 04:19:07.336853 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 04:19:07.336853 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 04:19:07.336853 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 04:19:07.336853 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 04:19:07.336853 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 04:19:07.336853 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 04:19:07.336853 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 04:19:07.336853 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 04:19:07.336853 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 04:19:07.338073 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 04:19:07.338073 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 04:19:07.338073 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 04:19:07.338073 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 04:19:07.338073 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 04:19:07.338073 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 04:19:07.338073 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 04:19:07.338073 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 04:19:07.338073 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 04:19:07.338073 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 04:19:07.338073 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 04:19:07.338073 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 04:19:07.338073 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 566/1149] 10.222.0.30 () {56 vars in 1096 bytes} [Tue Sep 6 04:19:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 04:19:07.342865 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bcf944f-9359-432d-a86a-64ca6ee06800 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:19:07.447122 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bcf944f-9359-432d-a86a-64ca6ee06800 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:19:07.480240 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045ee359-fb20-4e1e-af60-9e262b49d5ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:19:07.480841 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045ee359-fb20-4e1e-af60-9e262b49d5ac None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:19:07.481124 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045ee359-fb20-4e1e-af60-9e262b49d5ac None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:19:07.481426 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045ee359-fb20-4e1e-af60-9e262b49d5ac None admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:19:07.482093 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045ee359-fb20-4e1e-af60-9e262b49d5ac None admin] RBAC: Adding query filter params (name=admin) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:19:07.482398 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045ee359-fb20-4e1e-af60-9e262b49d5ac None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:19:07.483766 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045ee359-fb20-4e1e-af60-9e262b49d5ac None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:19:07.492163 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 584/1150] 10.222.0.30 () {56 vars in 1099 bytes} [Tue Sep 6 04:19:07 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:19:07.497999 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abf79cd-f36c-4106-ab40-6a712cc59a7a None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:19:07.526119 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:19:07.596582 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abf79cd-f36c-4106-ab40-6a712cc59a7a None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:19:07.630962 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a183af8e-25a6-45eb-8562-ac085c3188a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:19:07.631780 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a183af8e-25a6-45eb-8562-ac085c3188a0 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:19:07.632068 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a183af8e-25a6-45eb-8562-ac085c3188a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:19:07.632348 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a183af8e-25a6-45eb-8562-ac085c3188a0 None admin] PATH_INFO: `/v3/users/05ef446a47544e0189480d6729f0a0c6` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:19:07.633730 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a183af8e-25a6-45eb-8562-ac085c3188a0 None admin] RBAC: Authorizing `identity:get_user(user_id=05ef446a47544e0189480d6729f0a0c6)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:19:07.636355 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a183af8e-25a6-45eb-8562-ac085c3188a0 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:19:07.639144 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 567/1151] 10.222.0.30 () {56 vars in 1177 bytes} [Tue Sep 6 04:19:07 2022] GET /identity/v3/users/05ef446a47544e0189480d6729f0a0c6 => generated 286 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:19:07.645036 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045ee359-fb20-4e1e-af60-9e262b49d5ac None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:19:07.663999 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:19:07.745740 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045ee359-fb20-4e1e-af60-9e262b49d5ac None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:19:07.778517 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d87d99-79ef-4ab0-baed-eeefde18d941 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:19:07.779141 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d87d99-79ef-4ab0-baed-eeefde18d941 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:19:07.779415 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d87d99-79ef-4ab0-baed-eeefde18d941 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:19:07.779684 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d87d99-79ef-4ab0-baed-eeefde18d941 None admin] PATH_INFO: `/v3/projects/25d5bd72251b46cd85ff59790e022398` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:19:07.780791 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d87d99-79ef-4ab0-baed-eeefde18d941 None admin] RBAC: Authorizing `identity:get_project(project_id=25d5bd72251b46cd85ff59790e022398)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:19:07.783081 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d87d99-79ef-4ab0-baed-eeefde18d941 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:19:07.784694 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 585/1152] 10.222.0.30 () {56 vars in 1186 bytes} [Tue Sep 6 04:19:07 2022] GET /identity/v3/projects/25d5bd72251b46cd85ff59790e022398 => generated 311 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:19:07.790711 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a183af8e-25a6-45eb-8562-ac085c3188a0 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:19:07.880982 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:19:07.887400 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a183af8e-25a6-45eb-8562-ac085c3188a0 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:19:07.921662 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946e5ae5-4510-4855-87a5-8eab952a8dbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:19:07.923085 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946e5ae5-4510-4855-87a5-8eab952a8dbf None admin] REQUEST_METHOD: `PUT` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:19:07.923453 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946e5ae5-4510-4855-87a5-8eab952a8dbf None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:19:07.923725 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946e5ae5-4510-4855-87a5-8eab952a8dbf None admin] PATH_INFO: `/v3/projects/25d5bd72251b46cd85ff59790e022398/users/05ef446a47544e0189480d6729f0a0c6/roles/ea4c814713b149108e488de855b025e4` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:19:07.925646 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946e5ae5-4510-4855-87a5-8eab952a8dbf None admin] RBAC: Authorizing `identity:create_grant(project_id=25d5bd72251b46cd85ff59790e022398, user_id=05ef446a47544e0189480d6729f0a0c6, role_id=ea4c814713b149108e488de855b025e4)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:19:07.927059 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946e5ae5-4510-4855-87a5-8eab952a8dbf None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:19:08.003381 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 568/1153] 10.222.0.30 () {58 vars in 1439 bytes} [Tue Sep 6 04:19:07 2022] PUT /identity/v3/projects/25d5bd72251b46cd85ff59790e022398/users/05ef446a47544e0189480d6729f0a0c6/roles/ea4c814713b149108e488de855b025e4 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 04:19:09.985945 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c809bb9-79d6-452c-a98a-0bcedce58cb9 None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:19:09.986493 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c809bb9-79d6-452c-a98a-0bcedce58cb9 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:19:09.986493 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c809bb9-79d6-452c-a98a-0bcedce58cb9 None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:19:09.987815 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 586/1154] 10.222.0.30 () {54 vars in 927 bytes} [Tue Sep 6 04:19:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 04:19:09.992885 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd52317-7f0a-4e4a-a60b-57ec56accbee None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:19:09.993327 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd52317-7f0a-4e4a-a60b-57ec56accbee None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:19:09.993470 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd52317-7f0a-4e4a-a60b-57ec56accbee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:19:10.012164 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-3bd52317-7f0a-4e4a-a60b-57ec56accbee None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:19:10.013364 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-3bd52317-7f0a-4e4a-a60b-57ec56accbee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:19:10.078108 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 569/1155] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Sep 6 04:19:09 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 06 04:19:10.352516 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c809bb9-79d6-452c-a98a-0bcedce58cb9 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:19:10.435644 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:19:10.452047 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c809bb9-79d6-452c-a98a-0bcedce58cb9 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:19:10.483571 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f5cd8e-920a-4da7-b142-058556348fc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:19:10.484181 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f5cd8e-920a-4da7-b142-058556348fc8 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:19:10.484994 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f5cd8e-920a-4da7-b142-058556348fc8 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:19:10.485258 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f5cd8e-920a-4da7-b142-058556348fc8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:19:10.495129 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f5cd8e-920a-4da7-b142-058556348fc8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 04:19:10.499913 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-65f5cd8e-920a-4da7-b142-058556348fc8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 04:19:10.499913 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 04:19:10.499913 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 04:19:10.499913 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 04:19:10.499913 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 04:19:10.499913 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 04:19:10.499913 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 04:19:10.499913 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 04:19:10.499913 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 04:19:10.499913 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 04:19:10.499913 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 04:19:10.499913 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 04:19:10.499913 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 04:19:10.499913 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 04:19:10.499913 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 04:19:10.499913 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 04:19:10.501930 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 04:19:10.501930 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 04:19:10.501930 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 04:19:10.501930 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 04:19:10.501930 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 04:19:10.501930 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 04:19:10.501930 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 04:19:10.501930 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 04:19:10.501930 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 04:19:10.501930 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 04:19:10.501930 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 04:19:10.501930 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 04:19:10.501930 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 04:19:10.501930 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 04:19:10.501930 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 04:19:10.501930 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 04:19:10.503122 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 04:19:10.503122 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 04:19:10.503122 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 04:19:10.503122 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 04:19:10.503122 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 04:19:10.503122 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 04:19:10.503122 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 04:19:10.503122 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 04:19:10.503122 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 04:19:10.503122 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 04:19:10.503122 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 04:19:10.503122 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 04:19:10.503122 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 587/1156] 10.222.0.30 () {56 vars in 1096 bytes} [Tue Sep 6 04:19:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 04:19:10.507985 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd52317-7f0a-4e4a-a60b-57ec56accbee None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:19:10.587433 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:19:10.602280 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd52317-7f0a-4e4a-a60b-57ec56accbee None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:19:10.636039 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acea823b-9913-4843-9d34-6448a26640b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:19:10.636576 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acea823b-9913-4843-9d34-6448a26640b0 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:19:10.636852 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acea823b-9913-4843-9d34-6448a26640b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:19:10.637120 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acea823b-9913-4843-9d34-6448a26640b0 None admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:19:10.637802 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acea823b-9913-4843-9d34-6448a26640b0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:19:10.638102 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acea823b-9913-4843-9d34-6448a26640b0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:19:10.639490 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acea823b-9913-4843-9d34-6448a26640b0 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:19:10.647439 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 570/1157] 10.222.0.30 () {56 vars in 1099 bytes} [Tue Sep 6 04:19:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:19:10.653163 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f5cd8e-920a-4da7-b142-058556348fc8 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:19:10.751220 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f5cd8e-920a-4da7-b142-058556348fc8 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:19:10.784131 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90953da-cf6e-4630-95fb-3a5de2c59e1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:19:10.784745 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90953da-cf6e-4630-95fb-3a5de2c59e1e None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:19:10.785022 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90953da-cf6e-4630-95fb-3a5de2c59e1e None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:19:10.785291 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90953da-cf6e-4630-95fb-3a5de2c59e1e None admin] PATH_INFO: `/v3/users/05ef446a47544e0189480d6729f0a0c6` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:19:10.786419 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90953da-cf6e-4630-95fb-3a5de2c59e1e None admin] RBAC: Authorizing `identity:get_user(user_id=05ef446a47544e0189480d6729f0a0c6)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:19:10.787839 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90953da-cf6e-4630-95fb-3a5de2c59e1e None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:19:10.789413 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 588/1158] 10.222.0.30 () {56 vars in 1177 bytes} [Tue Sep 6 04:19:10 2022] GET /identity/v3/users/05ef446a47544e0189480d6729f0a0c6 => generated 286 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:19:10.795372 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acea823b-9913-4843-9d34-6448a26640b0 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:19:10.893352 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acea823b-9913-4843-9d34-6448a26640b0 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:19:10.925835 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20f2358-e3ed-425f-9623-7a8a76102e68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:19:10.926433 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20f2358-e3ed-425f-9623-7a8a76102e68 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:19:10.926711 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20f2358-e3ed-425f-9623-7a8a76102e68 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:19:10.926978 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20f2358-e3ed-425f-9623-7a8a76102e68 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:19:10.941929 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:19:10.992336 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f20f2358-e3ed-425f-9623-7a8a76102e68 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:19:10.993810 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f20f2358-e3ed-425f-9623-7a8a76102e68 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:19:11.089115 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 571/1159] 10.222.0.30 () {58 vars in 1282 bytes} [Tue Sep 6 04:19:10 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 06 04:19:11.095905 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90953da-cf6e-4630-95fb-3a5de2c59e1e None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:19:11.110841 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:19:11.193322 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90953da-cf6e-4630-95fb-3a5de2c59e1e None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:19:11.226530 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd5819c-4d30-4857-bb13-dbbe4fb6eeb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:19:11.227141 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd5819c-4d30-4857-bb13-dbbe4fb6eeb0 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:19:11.227415 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd5819c-4d30-4857-bb13-dbbe4fb6eeb0 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:19:11.227682 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd5819c-4d30-4857-bb13-dbbe4fb6eeb0 None admin] PATH_INFO: `/v3/projects/25d5bd72251b46cd85ff59790e022398` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:19:11.228776 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd5819c-4d30-4857-bb13-dbbe4fb6eeb0 None admin] RBAC: Authorizing `identity:get_project(project_id=25d5bd72251b46cd85ff59790e022398)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:19:11.230221 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd5819c-4d30-4857-bb13-dbbe4fb6eeb0 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:19:11.231814 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 589/1160] 10.222.0.30 () {56 vars in 1186 bytes} [Tue Sep 6 04:19:11 2022] GET /identity/v3/projects/25d5bd72251b46cd85ff59790e022398 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:19:11.237944 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20f2358-e3ed-425f-9623-7a8a76102e68 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:19:11.283924 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:19:11.335244 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20f2358-e3ed-425f-9623-7a8a76102e68 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:19:11.368355 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c30a113-ee15-4042-9e9c-f93f99a646cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:19:11.368952 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c30a113-ee15-4042-9e9c-f93f99a646cb None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:19:11.369229 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c30a113-ee15-4042-9e9c-f93f99a646cb None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:19:11.369529 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c30a113-ee15-4042-9e9c-f93f99a646cb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:19:11.370251 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c30a113-ee15-4042-9e9c-f93f99a646cb None admin] RBAC: Adding query filter params (role.id=ea4c814713b149108e488de855b025e4, scope.project.id=25d5bd72251b46cd85ff59790e022398, user.id=05ef446a47544e0189480d6729f0a0c6) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:19:11.370558 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c30a113-ee15-4042-9e9c-f93f99a646cb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:19:11.371973 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c30a113-ee15-4042-9e9c-f93f99a646cb None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:19:11.378277 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 572/1161] 10.222.0.30 () {56 vars in 1374 bytes} [Tue Sep 6 04:19:11 2022] GET /identity/v3/role_assignments?user.id=05ef446a47544e0189480d6729f0a0c6&scope.project.id=25d5bd72251b46cd85ff59790e022398&role.id=ea4c814713b149108e488de855b025e4 => generated 608 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:19:13.298215 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0a7333-2aeb-4c56-a562-41ae55be3775 None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:19:13.298746 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0a7333-2aeb-4c56-a562-41ae55be3775 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:19:13.298746 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0a7333-2aeb-4c56-a562-41ae55be3775 None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:19:13.300078 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 590/1162] 10.222.0.30 () {54 vars in 927 bytes} [Tue Sep 6 04:19:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 04:19:13.305254 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868f0011-b01a-4ab9-abc9-1a5a3805ab60 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:19:13.305698 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868f0011-b01a-4ab9-abc9-1a5a3805ab60 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:19:13.305839 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868f0011-b01a-4ab9-abc9-1a5a3805ab60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:19:13.325169 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-868f0011-b01a-4ab9-abc9-1a5a3805ab60 None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:19:13.326402 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-868f0011-b01a-4ab9-abc9-1a5a3805ab60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:19:13.391591 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 573/1163] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Sep 6 04:19:13 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 04:19:13.654900 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0a7333-2aeb-4c56-a562-41ae55be3775 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:19:13.733944 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:19:13.753515 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0a7333-2aeb-4c56-a562-41ae55be3775 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:19:13.791746 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c05bcf1-df79-48c7-ad9e-77413a3ead4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:19:13.792360 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c05bcf1-df79-48c7-ad9e-77413a3ead4e None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:19:13.792642 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c05bcf1-df79-48c7-ad9e-77413a3ead4e None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:19:13.792914 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c05bcf1-df79-48c7-ad9e-77413a3ead4e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:19:13.858153 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c05bcf1-df79-48c7-ad9e-77413a3ead4e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:19:13.860163 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c05bcf1-df79-48c7-ad9e-77413a3ead4e None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:19:13.934914 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:19:13.954445 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 591/1164] 10.222.0.30 () {58 vars in 1282 bytes} [Tue Sep 6 04:19:13 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 06 04:19:13.960497 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868f0011-b01a-4ab9-abc9-1a5a3805ab60 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:19:14.003122 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:19:14.060396 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868f0011-b01a-4ab9-abc9-1a5a3805ab60 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:19:14.096157 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca26d12-19d9-4149-b6c4-9016982b1312 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:19:14.096789 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca26d12-19d9-4149-b6c4-9016982b1312 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:19:14.097063 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca26d12-19d9-4149-b6c4-9016982b1312 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:19:14.097370 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca26d12-19d9-4149-b6c4-9016982b1312 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:19:14.104430 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca26d12-19d9-4149-b6c4-9016982b1312 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:19:14.106743 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca26d12-19d9-4149-b6c4-9016982b1312 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:19:14.124155 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-6ca26d12-19d9-4149-b6c4-9016982b1312 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Sep 06 04:19:14.124155 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 04:19:14.124155 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Sep 06 04:19:14.124155 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Sep 06 04:19:14.124155 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 06 04:19:14.124155 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 06 04:19:14.124155 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 06 04:19:14.124155 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 06 04:19:14.124155 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Sep 06 04:19:14.124155 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 04:19:14.124155 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 06 04:19:14.124155 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 04:19:14.124155 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 04:19:14.124155 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 04:19:14.124155 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 04:19:14.124155 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 04:19:14.124155 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 04:19:14.130829 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 04:19:14.130829 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 04:19:14.130829 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 04:19:14.130829 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 04:19:14.130829 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 04:19:14.130829 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 04:19:14.130829 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 04:19:14.130829 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 04:19:14.130829 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 04:19:14.130829 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 04:19:14.130829 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 04:19:14.130829 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 04:19:14.130829 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 04:19:14.130829 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 04:19:14.130829 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 04:19:14.130829 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 04:19:14.131900 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 04:19:14.131900 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 04:19:14.131900 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 04:19:14.131900 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 04:19:14.131900 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 04:19:14.131900 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 04:19:14.131900 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 04:19:14.131900 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 04:19:14.131900 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 04:19:14.131900 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 04:19:14.131900 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Sep 06 04:19:14.131900 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 04:19:14.131900 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Sep 06 04:19:14.131900 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 04:19:14.131900 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 574/1165] 10.222.0.30 () {56 vars in 1117 bytes} [Tue Sep 6 04:19:13 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 04:19:14.132912 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c05bcf1-df79-48c7-ad9e-77413a3ead4e None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:19:14.235848 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c05bcf1-df79-48c7-ad9e-77413a3ead4e None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:19:14.269320 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff69d0b-32f1-4b21-a224-f3d9275d6990 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:19:14.269932 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff69d0b-32f1-4b21-a224-f3d9275d6990 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:19:14.270210 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff69d0b-32f1-4b21-a224-f3d9275d6990 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:19:14.270482 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff69d0b-32f1-4b21-a224-f3d9275d6990 None admin] PATH_INFO: `/v3/domains` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:19:14.271118 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff69d0b-32f1-4b21-a224-f3d9275d6990 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:19:14.271415 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff69d0b-32f1-4b21-a224-f3d9275d6990 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:19:14.272779 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff69d0b-32f1-4b21-a224-f3d9275d6990 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:19:14.279687 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 592/1166] 10.222.0.30 () {56 vars in 1115 bytes} [Tue Sep 6 04:19:14 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:19:16.160401 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27c3376-439b-4549-94b4-05ea1c41382a None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:19:16.160959 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27c3376-439b-4549-94b4-05ea1c41382a None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:19:16.160959 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27c3376-439b-4549-94b4-05ea1c41382a None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:19:16.162318 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 575/1167] 10.222.0.30 () {54 vars in 927 bytes} [Tue Sep 6 04:19:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 04:19:16.167540 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06fc090-a2f4-4605-9209-d074e6850e04 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:19:16.167950 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06fc090-a2f4-4605-9209-d074e6850e04 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:19:16.168096 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06fc090-a2f4-4605-9209-d074e6850e04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:19:16.186726 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-e06fc090-a2f4-4605-9209-d074e6850e04 None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:19:16.188145 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-e06fc090-a2f4-4605-9209-d074e6850e04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:19:16.227981 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:19:16.254887 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 593/1168] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Sep 6 04:19:16 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 04:19:16.521798 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27c3376-439b-4549-94b4-05ea1c41382a None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:19:16.616365 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:19:16.620049 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27c3376-439b-4549-94b4-05ea1c41382a None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:19:16.653156 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec60743-e5e4-43b1-88d2-0723d7b128d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:19:16.653783 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec60743-e5e4-43b1-88d2-0723d7b128d2 None admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:19:16.654058 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec60743-e5e4-43b1-88d2-0723d7b128d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:19:16.654324 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec60743-e5e4-43b1-88d2-0723d7b128d2 None admin] PATH_INFO: `/v3/domains` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:19:16.654990 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec60743-e5e4-43b1-88d2-0723d7b128d2 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:19:16.657057 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec60743-e5e4-43b1-88d2-0723d7b128d2 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:19:16.727982 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 576/1169] 10.222.0.30 () {60 vars in 1138 bytes} [Tue Sep 6 04:19:16 2022] POST /identity/v3/domains => generated 267 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 04:19:18.653113 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7d62f3-99e0-45d1-931d-645dc7cb1849 None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:19:18.653645 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7d62f3-99e0-45d1-931d-645dc7cb1849 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:19:18.653645 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7d62f3-99e0-45d1-931d-645dc7cb1849 None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:19:18.654970 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 594/1170] 10.222.0.30 () {54 vars in 927 bytes} [Tue Sep 6 04:19:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 04:19:18.660155 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb003633-820f-4e4f-8aba-da738fe1db76 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:19:18.660581 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb003633-820f-4e4f-8aba-da738fe1db76 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:19:18.660681 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb003633-820f-4e4f-8aba-da738fe1db76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:19:18.679030 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-eb003633-820f-4e4f-8aba-da738fe1db76 None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:19:18.680203 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-eb003633-820f-4e4f-8aba-da738fe1db76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:19:18.744224 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 577/1171] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Sep 6 04:19:18 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 04:19:19.017012 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7d62f3-99e0-45d1-931d-645dc7cb1849 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:19:19.113793 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7d62f3-99e0-45d1-931d-645dc7cb1849 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:19:19.146654 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef746ca-4906-442b-a298-683c6ae27556 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:19:19.147262 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef746ca-4906-442b-a298-683c6ae27556 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:19:19.147537 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef746ca-4906-442b-a298-683c6ae27556 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:19:19.147804 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef746ca-4906-442b-a298-683c6ae27556 None admin] PATH_INFO: `/v3/domains/3378466dfa7f4e258b92e095bf92c933` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:19:19.158284 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef746ca-4906-442b-a298-683c6ae27556 None admin] RBAC: Authorizing `identity:get_domain(domain_id=3378466dfa7f4e258b92e095bf92c933)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:19:19.159715 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef746ca-4906-442b-a298-683c6ae27556 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:19:19.161278 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 595/1172] 10.222.0.30 () {56 vars in 1183 bytes} [Tue Sep 6 04:19:19 2022] GET /identity/v3/domains/3378466dfa7f4e258b92e095bf92c933 => generated 267 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:19:19.166974 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb003633-820f-4e4f-8aba-da738fe1db76 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:19:19.256440 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:19:19.265689 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb003633-820f-4e4f-8aba-da738fe1db76 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:19:19.298681 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85eb48e-7903-4976-b758-23a5d0741d79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:19:19.299288 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85eb48e-7903-4976-b758-23a5d0741d79 None admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:19:19.299569 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85eb48e-7903-4976-b758-23a5d0741d79 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:19:19.299840 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85eb48e-7903-4976-b758-23a5d0741d79 None admin] PATH_INFO: `/v3/projects` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:19:19.300492 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f85eb48e-7903-4976-b758-23a5d0741d79 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:19:19.301953 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f85eb48e-7903-4976-b758-23a5d0741d79 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:19:19.412137 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 578/1173] 10.222.0.30 () {60 vars in 1141 bytes} [Tue Sep 6 04:19:19 2022] POST /identity/v3/projects => generated 361 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 04:19:21.366407 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e306db9-d130-426a-92b5-d2d842860da5 None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:19:21.366927 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e306db9-d130-426a-92b5-d2d842860da5 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:19:21.366927 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e306db9-d130-426a-92b5-d2d842860da5 None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:19:21.368344 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 596/1174] 10.222.0.30 () {54 vars in 927 bytes} [Tue Sep 6 04:19:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 04:19:21.373418 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35750844-63ca-4cea-977a-9d1c87cf16ca None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:19:21.373811 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35750844-63ca-4cea-977a-9d1c87cf16ca None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:19:21.373963 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35750844-63ca-4cea-977a-9d1c87cf16ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:19:21.393877 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-35750844-63ca-4cea-977a-9d1c87cf16ca None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:19:21.395067 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-35750844-63ca-4cea-977a-9d1c87cf16ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:19:21.458419 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 579/1175] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Sep 6 04:19:21 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 04:19:21.755664 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e306db9-d130-426a-92b5-d2d842860da5 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:19:21.791404 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:19:21.857127 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e306db9-d130-426a-92b5-d2d842860da5 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:19:21.890822 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae31471-a36a-48ac-9a26-b73a0dc26107 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:19:21.891444 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae31471-a36a-48ac-9a26-b73a0dc26107 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:19:21.891725 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae31471-a36a-48ac-9a26-b73a0dc26107 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:19:21.891996 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae31471-a36a-48ac-9a26-b73a0dc26107 None admin] PATH_INFO: `/v3/domains/3378466dfa7f4e258b92e095bf92c933` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:19:21.893097 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae31471-a36a-48ac-9a26-b73a0dc26107 None admin] RBAC: Authorizing `identity:get_domain(domain_id=3378466dfa7f4e258b92e095bf92c933)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:19:21.894564 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae31471-a36a-48ac-9a26-b73a0dc26107 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:19:21.896119 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 597/1176] 10.222.0.30 () {56 vars in 1183 bytes} [Tue Sep 6 04:19:21 2022] GET /identity/v3/domains/3378466dfa7f4e258b92e095bf92c933 => generated 267 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:19:21.901945 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35750844-63ca-4cea-977a-9d1c87cf16ca None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:19:21.971974 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:19:22.000850 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35750844-63ca-4cea-977a-9d1c87cf16ca None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:19:22.034742 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20accb5a-1683-4ba9-bc28-9287831be2f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:19:22.035489 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20accb5a-1683-4ba9-bc28-9287831be2f1 None admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:19:22.035935 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20accb5a-1683-4ba9-bc28-9287831be2f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:19:22.036634 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20accb5a-1683-4ba9-bc28-9287831be2f1 None admin] PATH_INFO: `/v3/users` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:19:22.037290 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20accb5a-1683-4ba9-bc28-9287831be2f1 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:19:22.038742 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20accb5a-1683-4ba9-bc28-9287831be2f1 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:19:22.141377 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 580/1177] 10.222.0.30 () {60 vars in 1132 bytes} [Tue Sep 6 04:19:21 2022] POST /identity/v3/users => generated 311 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 04:19:24.156440 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee1d259-973b-4aed-b55e-60a016e9b005 None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:19:24.157056 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee1d259-973b-4aed-b55e-60a016e9b005 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:19:24.157056 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee1d259-973b-4aed-b55e-60a016e9b005 None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:19:24.158382 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 598/1178] 10.222.0.30 () {54 vars in 927 bytes} [Tue Sep 6 04:19:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 04:19:24.163431 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965c949e-a40f-42c6-99b7-0aa361340a07 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:19:24.163798 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965c949e-a40f-42c6-99b7-0aa361340a07 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:19:24.163976 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965c949e-a40f-42c6-99b7-0aa361340a07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:19:24.181553 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-965c949e-a40f-42c6-99b7-0aa361340a07 None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:19:24.182727 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-965c949e-a40f-42c6-99b7-0aa361340a07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:19:24.253454 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 581/1179] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Sep 6 04:19:24 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 04:19:24.544211 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee1d259-973b-4aed-b55e-60a016e9b005 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:19:24.647984 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee1d259-973b-4aed-b55e-60a016e9b005 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:19:24.649524 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:19:24.681349 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20113ec3-32b7-4807-9c0a-55825f8c52a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:19:24.681976 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20113ec3-32b7-4807-9c0a-55825f8c52a8 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:19:24.682250 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20113ec3-32b7-4807-9c0a-55825f8c52a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:19:24.682517 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20113ec3-32b7-4807-9c0a-55825f8c52a8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:19:24.693129 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20113ec3-32b7-4807-9c0a-55825f8c52a8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 04:19:24.698191 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-20113ec3-32b7-4807-9c0a-55825f8c52a8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 04:19:24.698191 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 04:19:24.698191 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 04:19:24.698191 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 04:19:24.698191 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 04:19:24.698191 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 04:19:24.698191 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 04:19:24.698191 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 04:19:24.698191 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 04:19:24.698191 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 04:19:24.698191 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 04:19:24.698191 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 04:19:24.698191 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 04:19:24.698191 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 04:19:24.698191 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 04:19:24.698191 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 04:19:24.699259 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 04:19:24.699259 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 04:19:24.699259 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 04:19:24.699259 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 04:19:24.699259 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 04:19:24.699259 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 04:19:24.699259 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 04:19:24.699259 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 04:19:24.699259 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 04:19:24.699259 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 04:19:24.699259 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 04:19:24.699259 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 04:19:24.699259 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 04:19:24.699259 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 04:19:24.699259 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 04:19:24.699259 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 04:19:24.700273 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 04:19:24.700273 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 04:19:24.700273 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 04:19:24.700273 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 04:19:24.700273 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 04:19:24.700273 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 04:19:24.700273 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 04:19:24.700273 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 04:19:24.700273 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 04:19:24.700273 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 04:19:24.700273 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 04:19:24.700273 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 04:19:24.700273 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 599/1180] 10.222.0.30 () {56 vars in 1096 bytes} [Tue Sep 6 04:19:24 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 04:19:24.705801 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965c949e-a40f-42c6-99b7-0aa361340a07 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:19:24.761971 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:19:24.801999 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965c949e-a40f-42c6-99b7-0aa361340a07 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:19:24.835564 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01b8865-4e8a-496a-9241-d67f83f32207 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:19:24.836163 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01b8865-4e8a-496a-9241-d67f83f32207 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:19:24.836445 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01b8865-4e8a-496a-9241-d67f83f32207 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:19:24.836718 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01b8865-4e8a-496a-9241-d67f83f32207 None admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:19:24.837407 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01b8865-4e8a-496a-9241-d67f83f32207 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:19:24.837714 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01b8865-4e8a-496a-9241-d67f83f32207 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:19:24.839079 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01b8865-4e8a-496a-9241-d67f83f32207 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:19:24.847030 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 582/1181] 10.222.0.30 () {56 vars in 1099 bytes} [Tue Sep 6 04:19:24 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:19:24.852729 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20113ec3-32b7-4807-9c0a-55825f8c52a8 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:19:24.953792 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20113ec3-32b7-4807-9c0a-55825f8c52a8 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:19:24.988538 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35607521-d0e8-42a7-ac5e-2c30eb242b00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:19:24.989162 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35607521-d0e8-42a7-ac5e-2c30eb242b00 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:19:24.989476 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35607521-d0e8-42a7-ac5e-2c30eb242b00 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:19:24.989751 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35607521-d0e8-42a7-ac5e-2c30eb242b00 None admin] PATH_INFO: `/v3/users/87fb01cd63f848db80bf8a8e3a59e38c` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:19:25.002988 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35607521-d0e8-42a7-ac5e-2c30eb242b00 None admin] RBAC: Authorizing `identity:get_user(user_id=87fb01cd63f848db80bf8a8e3a59e38c)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:19:25.004437 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35607521-d0e8-42a7-ac5e-2c30eb242b00 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:19:25.006077 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 600/1182] 10.222.0.30 () {56 vars in 1177 bytes} [Tue Sep 6 04:19:24 2022] GET /identity/v3/users/87fb01cd63f848db80bf8a8e3a59e38c => generated 311 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:19:25.012116 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01b8865-4e8a-496a-9241-d67f83f32207 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:19:25.107897 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01b8865-4e8a-496a-9241-d67f83f32207 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:19:25.127209 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:19:25.140412 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f704eda2-10f1-41a5-9ee5-9fa66e8f2ec5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:19:25.141155 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f704eda2-10f1-41a5-9ee5-9fa66e8f2ec5 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:19:25.141464 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f704eda2-10f1-41a5-9ee5-9fa66e8f2ec5 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:19:25.141736 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f704eda2-10f1-41a5-9ee5-9fa66e8f2ec5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:19:25.205142 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f704eda2-10f1-41a5-9ee5-9fa66e8f2ec5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:19:25.206597 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f704eda2-10f1-41a5-9ee5-9fa66e8f2ec5 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:19:25.306283 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 583/1183] 10.222.0.30 () {58 vars in 1282 bytes} [Tue Sep 6 04:19:25 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 06 04:19:25.312879 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35607521-d0e8-42a7-ac5e-2c30eb242b00 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:19:25.333940 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:19:25.410317 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35607521-d0e8-42a7-ac5e-2c30eb242b00 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:19:25.443187 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921d3832-06d4-454f-856c-7ce82cfcabcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:19:25.443805 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921d3832-06d4-454f-856c-7ce82cfcabcf None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:19:25.444080 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921d3832-06d4-454f-856c-7ce82cfcabcf None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:19:25.444347 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921d3832-06d4-454f-856c-7ce82cfcabcf None admin] PATH_INFO: `/v3/projects/dc483cfba56e460fb029eb75cd9bc0d6` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:19:25.445471 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921d3832-06d4-454f-856c-7ce82cfcabcf None admin] RBAC: Authorizing `identity:get_project(project_id=dc483cfba56e460fb029eb75cd9bc0d6)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:19:25.446889 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921d3832-06d4-454f-856c-7ce82cfcabcf None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:19:25.448498 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 601/1184] 10.222.0.30 () {56 vars in 1186 bytes} [Tue Sep 6 04:19:25 2022] GET /identity/v3/projects/dc483cfba56e460fb029eb75cd9bc0d6 => generated 361 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:19:25.454954 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f704eda2-10f1-41a5-9ee5-9fa66e8f2ec5 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:19:25.476188 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:19:25.561178 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f704eda2-10f1-41a5-9ee5-9fa66e8f2ec5 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:19:25.594415 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6900dc05-2383-478e-b5e6-850880d28c61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:19:25.595008 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6900dc05-2383-478e-b5e6-850880d28c61 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:19:25.595284 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6900dc05-2383-478e-b5e6-850880d28c61 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:19:25.595551 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6900dc05-2383-478e-b5e6-850880d28c61 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:19:25.596267 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6900dc05-2383-478e-b5e6-850880d28c61 None admin] RBAC: Adding query filter params (role.id=ea4c814713b149108e488de855b025e4, scope.project.id=dc483cfba56e460fb029eb75cd9bc0d6, user.id=87fb01cd63f848db80bf8a8e3a59e38c) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:19:25.596571 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6900dc05-2383-478e-b5e6-850880d28c61 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:19:25.598017 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6900dc05-2383-478e-b5e6-850880d28c61 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:19:25.603177 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 584/1185] 10.222.0.30 () {56 vars in 1374 bytes} [Tue Sep 6 04:19:25 2022] GET /identity/v3/role_assignments?user.id=87fb01cd63f848db80bf8a8e3a59e38c&scope.project.id=dc483cfba56e460fb029eb75cd9bc0d6&role.id=ea4c814713b149108e488de855b025e4 => generated 259 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:19:27.474933 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de929fb5-cf86-4dc1-aad0-8bcd76f4f237 None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:19:27.475453 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de929fb5-cf86-4dc1-aad0-8bcd76f4f237 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:19:27.475453 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de929fb5-cf86-4dc1-aad0-8bcd76f4f237 None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:19:27.476791 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 602/1186] 10.222.0.30 () {54 vars in 927 bytes} [Tue Sep 6 04:19:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 06 04:19:27.481782 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57aed876-1e1a-47c6-ac5e-fe56071610d2 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:19:27.482176 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57aed876-1e1a-47c6-ac5e-fe56071610d2 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:19:27.482328 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57aed876-1e1a-47c6-ac5e-fe56071610d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:19:27.500472 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-57aed876-1e1a-47c6-ac5e-fe56071610d2 None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:19:27.501937 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-57aed876-1e1a-47c6-ac5e-fe56071610d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:19:27.574723 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 585/1187] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Sep 6 04:19:27 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 04:19:27.844362 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de929fb5-cf86-4dc1-aad0-8bcd76f4f237 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:19:27.930623 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:19:27.943472 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de929fb5-cf86-4dc1-aad0-8bcd76f4f237 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:19:27.976222 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18a136c-4603-4c8f-bd99-8d0fc0b1785e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:19:27.976833 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18a136c-4603-4c8f-bd99-8d0fc0b1785e None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:19:27.977106 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18a136c-4603-4c8f-bd99-8d0fc0b1785e None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:19:27.977400 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18a136c-4603-4c8f-bd99-8d0fc0b1785e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:19:27.987308 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c18a136c-4603-4c8f-bd99-8d0fc0b1785e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 04:19:27.991976 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-c18a136c-4603-4c8f-bd99-8d0fc0b1785e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 04:19:27.991976 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 04:19:27.991976 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 04:19:27.991976 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 04:19:27.991976 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 04:19:27.991976 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 04:19:27.991976 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 04:19:27.991976 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 04:19:27.991976 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 04:19:27.991976 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 04:19:27.991976 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 04:19:27.991976 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 04:19:27.991976 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 04:19:27.991976 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 04:19:27.991976 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 04:19:27.991976 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 04:19:27.993137 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 04:19:27.993137 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 04:19:27.993137 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 04:19:27.993137 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 04:19:27.993137 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 04:19:27.993137 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 04:19:27.993137 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 04:19:27.993137 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 04:19:27.993137 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 04:19:27.993137 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 04:19:27.993137 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 04:19:27.993137 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 04:19:27.993137 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 04:19:27.993137 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 04:19:27.993137 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 04:19:27.993137 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 04:19:27.994126 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 04:19:27.994126 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 04:19:27.994126 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 04:19:27.994126 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 04:19:27.994126 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 04:19:27.994126 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 04:19:27.994126 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 04:19:27.994126 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 04:19:27.994126 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 04:19:27.994126 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 04:19:27.994126 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 04:19:27.994126 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 04:19:27.994126 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 603/1188] 10.222.0.30 () {56 vars in 1096 bytes} [Tue Sep 6 04:19:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 04:19:27.999052 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57aed876-1e1a-47c6-ac5e-fe56071610d2 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:19:28.014521 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:19:28.095515 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57aed876-1e1a-47c6-ac5e-fe56071610d2 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:19:28.127905 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00aabc32-0ae0-4764-bf06-bd7462ebd09a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:19:28.128492 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00aabc32-0ae0-4764-bf06-bd7462ebd09a None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:19:28.128769 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00aabc32-0ae0-4764-bf06-bd7462ebd09a None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:19:28.129033 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00aabc32-0ae0-4764-bf06-bd7462ebd09a None admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:19:28.129707 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00aabc32-0ae0-4764-bf06-bd7462ebd09a None admin] RBAC: Adding query filter params (name=admin) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:19:28.130006 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00aabc32-0ae0-4764-bf06-bd7462ebd09a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:19:28.131345 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00aabc32-0ae0-4764-bf06-bd7462ebd09a None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:19:28.138976 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 586/1189] 10.222.0.30 () {56 vars in 1099 bytes} [Tue Sep 6 04:19:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:19:28.145394 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18a136c-4603-4c8f-bd99-8d0fc0b1785e None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:19:28.241247 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18a136c-4603-4c8f-bd99-8d0fc0b1785e None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:19:28.278734 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5627133-ec9f-4b00-8b63-1bb6656974af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:19:28.279353 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5627133-ec9f-4b00-8b63-1bb6656974af None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:19:28.279631 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5627133-ec9f-4b00-8b63-1bb6656974af None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:19:28.279901 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5627133-ec9f-4b00-8b63-1bb6656974af None admin] PATH_INFO: `/v3/users/87fb01cd63f848db80bf8a8e3a59e38c` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:19:28.281573 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5627133-ec9f-4b00-8b63-1bb6656974af None admin] RBAC: Authorizing `identity:get_user(user_id=87fb01cd63f848db80bf8a8e3a59e38c)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:19:28.283176 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5627133-ec9f-4b00-8b63-1bb6656974af None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:19:28.284746 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 604/1190] 10.222.0.30 () {56 vars in 1177 bytes} [Tue Sep 6 04:19:28 2022] GET /identity/v3/users/87fb01cd63f848db80bf8a8e3a59e38c => generated 311 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:19:28.290779 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00aabc32-0ae0-4764-bf06-bd7462ebd09a None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:19:28.366742 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:19:28.389290 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00aabc32-0ae0-4764-bf06-bd7462ebd09a None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:19:28.423422 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab8e4cf-26bc-4cf6-b600-263f6b81db58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:19:28.424443 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab8e4cf-26bc-4cf6-b600-263f6b81db58 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:19:28.424443 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab8e4cf-26bc-4cf6-b600-263f6b81db58 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:19:28.424641 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab8e4cf-26bc-4cf6-b600-263f6b81db58 None admin] PATH_INFO: `/v3/projects/dc483cfba56e460fb029eb75cd9bc0d6` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:19:28.425759 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab8e4cf-26bc-4cf6-b600-263f6b81db58 None admin] RBAC: Authorizing `identity:get_project(project_id=dc483cfba56e460fb029eb75cd9bc0d6)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:19:28.427176 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab8e4cf-26bc-4cf6-b600-263f6b81db58 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:19:28.428827 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 587/1191] 10.222.0.30 () {56 vars in 1186 bytes} [Tue Sep 6 04:19:28 2022] GET /identity/v3/projects/dc483cfba56e460fb029eb75cd9bc0d6 => generated 361 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:19:28.434860 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5627133-ec9f-4b00-8b63-1bb6656974af None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:19:28.453685 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:19:28.534487 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5627133-ec9f-4b00-8b63-1bb6656974af None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:19:28.566431 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f0224f-bad2-4b6a-9abb-81548d98fda6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:19:28.567081 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f0224f-bad2-4b6a-9abb-81548d98fda6 None admin] REQUEST_METHOD: `PUT` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:19:28.567363 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f0224f-bad2-4b6a-9abb-81548d98fda6 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:19:28.567639 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f0224f-bad2-4b6a-9abb-81548d98fda6 None admin] PATH_INFO: `/v3/projects/dc483cfba56e460fb029eb75cd9bc0d6/users/87fb01cd63f848db80bf8a8e3a59e38c/roles/ea4c814713b149108e488de855b025e4` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:19:28.569553 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f0224f-bad2-4b6a-9abb-81548d98fda6 None admin] RBAC: Authorizing `identity:create_grant(project_id=dc483cfba56e460fb029eb75cd9bc0d6, user_id=87fb01cd63f848db80bf8a8e3a59e38c, role_id=ea4c814713b149108e488de855b025e4)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:19:28.571080 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f0224f-bad2-4b6a-9abb-81548d98fda6 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:19:28.633013 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 605/1192] 10.222.0.30 () {58 vars in 1439 bytes} [Tue Sep 6 04:19:28 2022] PUT /identity/v3/projects/dc483cfba56e460fb029eb75cd9bc0d6/users/87fb01cd63f848db80bf8a8e3a59e38c/roles/ea4c814713b149108e488de855b025e4 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 04:19:30.536871 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524d77c4-0b22-4db3-822a-f460c54ce46b None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:19:30.537415 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524d77c4-0b22-4db3-822a-f460c54ce46b None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:19:30.537415 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524d77c4-0b22-4db3-822a-f460c54ce46b None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:19:30.538772 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 588/1193] 10.222.0.30 () {54 vars in 927 bytes} [Tue Sep 6 04:19:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 04:19:30.543924 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20a1b65-1088-4921-98ce-017418b49512 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:19:30.544337 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20a1b65-1088-4921-98ce-017418b49512 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:19:30.544466 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20a1b65-1088-4921-98ce-017418b49512 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:19:30.563511 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-c20a1b65-1088-4921-98ce-017418b49512 None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:19:30.564698 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-c20a1b65-1088-4921-98ce-017418b49512 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:19:30.632709 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 606/1194] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Sep 6 04:19:30 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 04:19:30.904407 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524d77c4-0b22-4db3-822a-f460c54ce46b None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:19:31.003697 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524d77c4-0b22-4db3-822a-f460c54ce46b None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:19:31.035878 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae761bc9-7840-453f-98c6-9a7b322e5b62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:19:31.036518 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae761bc9-7840-453f-98c6-9a7b322e5b62 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:19:31.036805 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae761bc9-7840-453f-98c6-9a7b322e5b62 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:19:31.037085 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae761bc9-7840-453f-98c6-9a7b322e5b62 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:19:31.048834 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae761bc9-7840-453f-98c6-9a7b322e5b62 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 04:19:31.053781 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-ae761bc9-7840-453f-98c6-9a7b322e5b62 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 04:19:31.053781 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 04:19:31.053781 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 04:19:31.053781 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 04:19:31.053781 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 04:19:31.053781 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 04:19:31.053781 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 04:19:31.053781 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 04:19:31.053781 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 04:19:31.053781 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 04:19:31.053781 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 04:19:31.053781 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 04:19:31.053781 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 04:19:31.053781 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 04:19:31.053781 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 04:19:31.053781 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 04:19:31.054878 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 04:19:31.054878 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 04:19:31.054878 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 04:19:31.054878 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 04:19:31.054878 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 04:19:31.054878 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 04:19:31.054878 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 04:19:31.054878 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 04:19:31.054878 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 04:19:31.054878 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 04:19:31.054878 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 04:19:31.054878 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 04:19:31.054878 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 04:19:31.054878 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 04:19:31.054878 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 04:19:31.054878 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 04:19:31.055766 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 04:19:31.055766 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 04:19:31.055766 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 04:19:31.055766 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 04:19:31.055766 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 04:19:31.055766 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 04:19:31.055766 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 04:19:31.055766 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 04:19:31.055766 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 04:19:31.055766 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 04:19:31.055766 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 04:19:31.055766 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 04:19:31.055766 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 589/1195] 10.222.0.30 () {56 vars in 1096 bytes} [Tue Sep 6 04:19:30 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 04:19:31.060808 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20a1b65-1088-4921-98ce-017418b49512 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:19:31.068825 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:19:31.156085 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20a1b65-1088-4921-98ce-017418b49512 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:19:31.188519 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3645b7c-fbb7-44fa-aedc-0fed35de846f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:19:31.189108 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3645b7c-fbb7-44fa-aedc-0fed35de846f None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:19:31.189412 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3645b7c-fbb7-44fa-aedc-0fed35de846f None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:19:31.189681 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3645b7c-fbb7-44fa-aedc-0fed35de846f None admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:19:31.190335 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3645b7c-fbb7-44fa-aedc-0fed35de846f None admin] RBAC: Adding query filter params (name=admin) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:19:31.190633 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3645b7c-fbb7-44fa-aedc-0fed35de846f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:19:31.191988 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3645b7c-fbb7-44fa-aedc-0fed35de846f None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:19:31.199673 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 607/1196] 10.222.0.30 () {56 vars in 1099 bytes} [Tue Sep 6 04:19:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:19:31.205472 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae761bc9-7840-453f-98c6-9a7b322e5b62 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:19:31.209191 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:19:31.303917 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae761bc9-7840-453f-98c6-9a7b322e5b62 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:19:31.335869 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea6361d-b6b9-4ed3-823d-e88ea54474d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:19:31.336486 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea6361d-b6b9-4ed3-823d-e88ea54474d2 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:19:31.336765 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea6361d-b6b9-4ed3-823d-e88ea54474d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:19:31.337035 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea6361d-b6b9-4ed3-823d-e88ea54474d2 None admin] PATH_INFO: `/v3/users/87fb01cd63f848db80bf8a8e3a59e38c` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:19:31.338165 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea6361d-b6b9-4ed3-823d-e88ea54474d2 None admin] RBAC: Authorizing `identity:get_user(user_id=87fb01cd63f848db80bf8a8e3a59e38c)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:19:31.339586 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea6361d-b6b9-4ed3-823d-e88ea54474d2 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:19:31.341143 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 590/1197] 10.222.0.30 () {56 vars in 1177 bytes} [Tue Sep 6 04:19:31 2022] GET /identity/v3/users/87fb01cd63f848db80bf8a8e3a59e38c => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:19:31.347623 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3645b7c-fbb7-44fa-aedc-0fed35de846f None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:19:31.412432 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:19:31.444229 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3645b7c-fbb7-44fa-aedc-0fed35de846f None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:19:31.476897 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47f6a74-a530-4136-b32c-fc2eaee156de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:19:31.477521 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47f6a74-a530-4136-b32c-fc2eaee156de None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:19:31.477798 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47f6a74-a530-4136-b32c-fc2eaee156de None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:19:31.478086 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47f6a74-a530-4136-b32c-fc2eaee156de None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:19:31.543510 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e47f6a74-a530-4136-b32c-fc2eaee156de None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:19:31.544949 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e47f6a74-a530-4136-b32c-fc2eaee156de None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:19:31.611248 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:19:31.645734 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 608/1198] 10.222.0.30 () {58 vars in 1282 bytes} [Tue Sep 6 04:19:31 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 06 04:19:31.651703 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea6361d-b6b9-4ed3-823d-e88ea54474d2 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:19:31.721866 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:19:31.750977 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea6361d-b6b9-4ed3-823d-e88ea54474d2 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:19:31.784765 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09df2149-10c0-4fa3-a460-753668fe7eb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:19:31.785419 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09df2149-10c0-4fa3-a460-753668fe7eb8 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:19:31.785701 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09df2149-10c0-4fa3-a460-753668fe7eb8 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:19:31.785973 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09df2149-10c0-4fa3-a460-753668fe7eb8 None admin] PATH_INFO: `/v3/projects/dc483cfba56e460fb029eb75cd9bc0d6` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:19:31.787081 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09df2149-10c0-4fa3-a460-753668fe7eb8 None admin] RBAC: Authorizing `identity:get_project(project_id=dc483cfba56e460fb029eb75cd9bc0d6)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:19:31.788496 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09df2149-10c0-4fa3-a460-753668fe7eb8 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:19:31.790148 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 591/1199] 10.222.0.30 () {56 vars in 1186 bytes} [Tue Sep 6 04:19:31 2022] GET /identity/v3/projects/dc483cfba56e460fb029eb75cd9bc0d6 => generated 361 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:19:31.796870 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47f6a74-a530-4136-b32c-fc2eaee156de None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:19:31.892624 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47f6a74-a530-4136-b32c-fc2eaee156de None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:19:31.924552 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf32de91-2db1-40e7-bdb2-f878836c978f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:19:31.925137 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf32de91-2db1-40e7-bdb2-f878836c978f None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:19:31.925441 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf32de91-2db1-40e7-bdb2-f878836c978f None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:19:31.925711 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf32de91-2db1-40e7-bdb2-f878836c978f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:19:31.926423 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf32de91-2db1-40e7-bdb2-f878836c978f None admin] RBAC: Adding query filter params (role.id=ea4c814713b149108e488de855b025e4, scope.project.id=dc483cfba56e460fb029eb75cd9bc0d6, user.id=87fb01cd63f848db80bf8a8e3a59e38c) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:19:31.926723 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf32de91-2db1-40e7-bdb2-f878836c978f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:19:31.928143 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf32de91-2db1-40e7-bdb2-f878836c978f None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:19:31.933637 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 609/1200] 10.222.0.30 () {56 vars in 1374 bytes} [Tue Sep 6 04:19:31 2022] GET /identity/v3/role_assignments?user.id=87fb01cd63f848db80bf8a8e3a59e38c&scope.project.id=dc483cfba56e460fb029eb75cd9bc0d6&role.id=ea4c814713b149108e488de855b025e4 => generated 608 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:19:41.781233 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=30 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:19:41.931076 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=84 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:20:44.984191 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edb9236-2334-4574-8189-e161369dfa12 None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:20:44.984803 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edb9236-2334-4574-8189-e161369dfa12 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:20:44.984803 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edb9236-2334-4574-8189-e161369dfa12 None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:20:44.986483 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 592/1201] 10.222.0.30 () {54 vars in 927 bytes} [Tue Sep 6 04:20:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 04:20:44.991536 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611abe3a-febe-4358-a7f5-0e92e6ac75e4 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:20:44.991934 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611abe3a-febe-4358-a7f5-0e92e6ac75e4 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:20:44.992134 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611abe3a-febe-4358-a7f5-0e92e6ac75e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:20:45.011051 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-611abe3a-febe-4358-a7f5-0e92e6ac75e4 None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:20:45.012247 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-611abe3a-febe-4358-a7f5-0e92e6ac75e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:20:45.078319 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 610/1202] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Sep 6 04:20:44 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 04:20:45.344211 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7edb9236-2334-4574-8189-e161369dfa12 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:20:45.445639 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7edb9236-2334-4574-8189-e161369dfa12 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:20:45.478395 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53333699-579f-4fd5-bdc2-5b0c3a2da9db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:20:45.479020 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53333699-579f-4fd5-bdc2-5b0c3a2da9db None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:20:45.479295 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53333699-579f-4fd5-bdc2-5b0c3a2da9db None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:20:45.479564 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53333699-579f-4fd5-bdc2-5b0c3a2da9db None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:20:45.489852 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53333699-579f-4fd5-bdc2-5b0c3a2da9db None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:20:45.491267 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53333699-579f-4fd5-bdc2-5b0c3a2da9db None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:20:45.505595 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-53333699-579f-4fd5-bdc2-5b0c3a2da9db None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 04:20:45.505595 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 04:20:45.505595 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 04:20:45.505595 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 04:20:45.505595 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 04:20:45.505595 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 04:20:45.505595 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 04:20:45.505595 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 04:20:45.505595 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 04:20:45.505595 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 04:20:45.505595 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 04:20:45.505595 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 04:20:45.505595 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 04:20:45.505595 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 04:20:45.505595 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 04:20:45.505595 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 04:20:45.506633 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 04:20:45.506633 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 04:20:45.506633 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 04:20:45.506633 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 04:20:45.506633 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 04:20:45.506633 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 04:20:45.506633 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 04:20:45.506633 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 04:20:45.506633 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 04:20:45.506633 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 04:20:45.506633 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 04:20:45.506633 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 04:20:45.506633 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 04:20:45.506633 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 04:20:45.506633 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 04:20:45.506633 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 04:20:45.506633 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 04:20:45.507751 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 04:20:45.507751 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 04:20:45.507751 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 04:20:45.507751 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 593/1203] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Sep 6 04:20:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 04:20:45.513165 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611abe3a-febe-4358-a7f5-0e92e6ac75e4 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:20:45.613620 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611abe3a-febe-4358-a7f5-0e92e6ac75e4 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:20:45.642956 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:20:45.648614 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3cde01e-4d46-4b30-bb0e-016aeb468913 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:20:45.649242 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cde01e-4d46-4b30-bb0e-016aeb468913 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:20:45.649527 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cde01e-4d46-4b30-bb0e-016aeb468913 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:20:45.649798 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cde01e-4d46-4b30-bb0e-016aeb468913 None admin] PATH_INFO: `/v3/domains` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:20:45.650447 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3cde01e-4d46-4b30-bb0e-016aeb468913 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:20:45.650739 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3cde01e-4d46-4b30-bb0e-016aeb468913 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:20:45.652102 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3cde01e-4d46-4b30-bb0e-016aeb468913 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:20:45.663299 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 611/1204] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Sep 6 04:20:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 04:20:45.669160 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53333699-579f-4fd5-bdc2-5b0c3a2da9db None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:20:45.720891 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:20:45.767357 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53333699-579f-4fd5-bdc2-5b0c3a2da9db None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:20:45.800221 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5f936e-0c7a-44f4-ad0e-7c2882a4f419 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:20:45.800827 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5f936e-0c7a-44f4-ad0e-7c2882a4f419 None admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:20:45.801102 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5f936e-0c7a-44f4-ad0e-7c2882a4f419 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:20:45.801395 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5f936e-0c7a-44f4-ad0e-7c2882a4f419 None admin] PATH_INFO: `/v3/users` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:20:45.802044 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5f936e-0c7a-44f4-ad0e-7c2882a4f419 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:20:45.803461 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5f936e-0c7a-44f4-ad0e-7c2882a4f419 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:20:45.896499 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 594/1205] 10.222.0.30 () {60 vars in 1132 bytes} [Tue Sep 6 04:20:45 2022] POST /identity/v3/users => generated 251 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 04:20:47.875399 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82dba35d-255d-4f56-92f8-4f2f23bf6162 None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:20:47.875948 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82dba35d-255d-4f56-92f8-4f2f23bf6162 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:20:47.875948 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82dba35d-255d-4f56-92f8-4f2f23bf6162 None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:20:47.878103 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 612/1206] 10.222.0.30 () {54 vars in 927 bytes} [Tue Sep 6 04:20:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 04:20:47.883180 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af4eab8-cda1-4cda-b3dd-28ac805be780 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:20:47.883602 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af4eab8-cda1-4cda-b3dd-28ac805be780 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:20:47.883732 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af4eab8-cda1-4cda-b3dd-28ac805be780 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:20:47.902208 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-8af4eab8-cda1-4cda-b3dd-28ac805be780 None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:20:47.903391 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-8af4eab8-cda1-4cda-b3dd-28ac805be780 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:20:47.970677 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 595/1207] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Sep 6 04:20:47 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 04:20:48.241534 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82dba35d-255d-4f56-92f8-4f2f23bf6162 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:20:48.341083 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82dba35d-255d-4f56-92f8-4f2f23bf6162 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:20:48.374304 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bcadec7-fb6d-41dc-916d-3c879cbdd2ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:20:48.374939 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcadec7-fb6d-41dc-916d-3c879cbdd2ed None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:20:48.375219 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcadec7-fb6d-41dc-916d-3c879cbdd2ed None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:20:48.375492 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcadec7-fb6d-41dc-916d-3c879cbdd2ed None admin] PATH_INFO: `/v3/roles/service` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:20:48.386751 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bcadec7-fb6d-41dc-916d-3c879cbdd2ed None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 04:20:48.391611 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-0bcadec7-fb6d-41dc-916d-3c879cbdd2ed None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 06 04:20:48.391611 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 04:20:48.391611 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 04:20:48.391611 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 04:20:48.391611 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 04:20:48.391611 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 04:20:48.391611 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 04:20:48.391611 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 04:20:48.391611 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 04:20:48.391611 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 04:20:48.391611 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 04:20:48.391611 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 04:20:48.391611 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 04:20:48.391611 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 04:20:48.391611 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 04:20:48.391611 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 04:20:48.392779 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 04:20:48.392779 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 04:20:48.392779 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 04:20:48.392779 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 04:20:48.392779 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 04:20:48.392779 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 04:20:48.392779 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 04:20:48.392779 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 04:20:48.392779 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 04:20:48.392779 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 04:20:48.392779 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 04:20:48.392779 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 04:20:48.392779 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 04:20:48.392779 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 04:20:48.392779 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 04:20:48.392779 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 04:20:48.393854 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 04:20:48.393854 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 04:20:48.393854 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 04:20:48.393854 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 04:20:48.393854 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 04:20:48.393854 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 04:20:48.393854 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 04:20:48.393854 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 04:20:48.393854 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 04:20:48.393854 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 04:20:48.393854 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 06 04:20:48.393854 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 04:20:48.394526 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 613/1208] 10.222.0.30 () {56 vars in 1102 bytes} [Tue Sep 6 04:20:48 2022] GET /identity/v3/roles/service => generated 85 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 04:20:48.400489 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af4eab8-cda1-4cda-b3dd-28ac805be780 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:20:48.443315 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:20:48.502771 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af4eab8-cda1-4cda-b3dd-28ac805be780 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:20:48.536618 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560f973c-418e-4ce2-8cff-3ae399aae8f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:20:48.537209 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560f973c-418e-4ce2-8cff-3ae399aae8f9 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:20:48.537512 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560f973c-418e-4ce2-8cff-3ae399aae8f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:20:48.537781 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560f973c-418e-4ce2-8cff-3ae399aae8f9 None admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:20:48.538436 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560f973c-418e-4ce2-8cff-3ae399aae8f9 None admin] RBAC: Adding query filter params (name=service) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:20:48.538735 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560f973c-418e-4ce2-8cff-3ae399aae8f9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:20:48.540117 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560f973c-418e-4ce2-8cff-3ae399aae8f9 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:20:48.547799 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 596/1209] 10.222.0.30 () {56 vars in 1103 bytes} [Tue Sep 6 04:20:48 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:20:48.554230 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bcadec7-fb6d-41dc-916d-3c879cbdd2ed None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:20:48.604770 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:20:48.653907 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bcadec7-fb6d-41dc-916d-3c879cbdd2ed None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:20:48.687886 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbc15bb-ebb6-4a90-b8dd-a4760d4f3cc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:20:48.688612 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbc15bb-ebb6-4a90-b8dd-a4760d4f3cc6 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:20:48.689199 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbc15bb-ebb6-4a90-b8dd-a4760d4f3cc6 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:20:48.689417 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbc15bb-ebb6-4a90-b8dd-a4760d4f3cc6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:20:48.700544 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bbc15bb-ebb6-4a90-b8dd-a4760d4f3cc6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:20:48.702037 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bbc15bb-ebb6-4a90-b8dd-a4760d4f3cc6 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:20:48.716711 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-1bbc15bb-ebb6-4a90-b8dd-a4760d4f3cc6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 04:20:48.716711 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 04:20:48.716711 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 04:20:48.716711 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 04:20:48.716711 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 04:20:48.716711 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 04:20:48.716711 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 04:20:48.716711 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 04:20:48.716711 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 04:20:48.716711 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 04:20:48.716711 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 04:20:48.716711 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 04:20:48.716711 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 04:20:48.716711 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 04:20:48.716711 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 04:20:48.716711 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 04:20:48.717774 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 04:20:48.717774 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 04:20:48.717774 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 04:20:48.717774 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 04:20:48.717774 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 04:20:48.717774 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 04:20:48.717774 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 04:20:48.717774 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 04:20:48.717774 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 04:20:48.717774 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 04:20:48.717774 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 04:20:48.717774 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 04:20:48.717774 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 04:20:48.717774 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 04:20:48.717774 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 04:20:48.717774 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 04:20:48.717774 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 04:20:48.718749 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 04:20:48.718749 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 04:20:48.718749 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 04:20:48.718749 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 614/1210] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Sep 6 04:20:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 04:20:48.724765 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560f973c-418e-4ce2-8cff-3ae399aae8f9 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:20:48.821552 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:20:48.824184 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560f973c-418e-4ce2-8cff-3ae399aae8f9 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:20:48.856805 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ea9f14-da14-4274-86da-59430cb1e3fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:20:48.857443 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ea9f14-da14-4274-86da-59430cb1e3fa None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:20:48.857720 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ea9f14-da14-4274-86da-59430cb1e3fa None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:20:48.857987 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ea9f14-da14-4274-86da-59430cb1e3fa None admin] PATH_INFO: `/v3/domains` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:20:48.858620 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ea9f14-da14-4274-86da-59430cb1e3fa None admin] RBAC: Adding query filter params (name=Default) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:20:48.858918 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ea9f14-da14-4274-86da-59430cb1e3fa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:20:48.860266 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ea9f14-da14-4274-86da-59430cb1e3fa None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:20:48.872872 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 597/1211] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Sep 6 04:20:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:20:48.878847 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbc15bb-ebb6-4a90-b8dd-a4760d4f3cc6 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:20:48.976384 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbc15bb-ebb6-4a90-b8dd-a4760d4f3cc6 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:20:48.977871 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:20:49.010163 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222d7331-918e-4aab-8589-75b5ca2dd9c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:20:49.010775 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222d7331-918e-4aab-8589-75b5ca2dd9c5 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:20:49.011047 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222d7331-918e-4aab-8589-75b5ca2dd9c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:20:49.011311 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222d7331-918e-4aab-8589-75b5ca2dd9c5 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:20:49.018148 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-222d7331-918e-4aab-8589-75b5ca2dd9c5 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:20:49.019570 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-222d7331-918e-4aab-8589-75b5ca2dd9c5 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:20:49.030649 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-222d7331-918e-4aab-8589-75b5ca2dd9c5 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 06 04:20:49.030649 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 04:20:49.030649 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 04:20:49.030649 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 04:20:49.030649 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 04:20:49.030649 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 04:20:49.030649 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 04:20:49.030649 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 04:20:49.030649 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 04:20:49.030649 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 04:20:49.030649 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 04:20:49.030649 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 04:20:49.030649 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 06 04:20:49.030649 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 06 04:20:49.030649 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 06 04:20:49.030649 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 06 04:20:49.036901 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 04:20:49.036901 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 04:20:49.036901 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 06 04:20:49.036901 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 04:20:49.036901 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 06 04:20:49.036901 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 04:20:49.036901 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 04:20:49.036901 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 04:20:49.036901 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 04:20:49.036901 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 04:20:49.036901 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 04:20:49.036901 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 04:20:49.036901 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 04:20:49.036901 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 04:20:49.036901 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 04:20:49.036901 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 04:20:49.036901 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 04:20:49.036901 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 04:20:49.038047 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 04:20:49.038047 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 04:20:49.038047 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 06 04:20:49.038047 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 06 04:20:49.038047 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 06 04:20:49.038047 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 06 04:20:49.038047 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 06 04:20:49.038047 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 06 04:20:49.038047 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 06 04:20:49.038047 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 04:20:49.038047 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 615/1212] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Sep 6 04:20:48 2022] GET /identity/v3/users/placement => generated 87 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 04:20:49.038767 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ea9f14-da14-4274-86da-59430cb1e3fa None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:20:49.135749 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ea9f14-da14-4274-86da-59430cb1e3fa None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:20:49.168443 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d7699e-3cb3-47cb-a5c0-6cf0ba280695 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:20:49.169044 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d7699e-3cb3-47cb-a5c0-6cf0ba280695 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:20:49.169342 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d7699e-3cb3-47cb-a5c0-6cf0ba280695 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:20:49.169623 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d7699e-3cb3-47cb-a5c0-6cf0ba280695 None admin] PATH_INFO: `/v3/users` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:20:49.170349 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d7699e-3cb3-47cb-a5c0-6cf0ba280695 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:20:49.170652 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d7699e-3cb3-47cb-a5c0-6cf0ba280695 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:20:49.172069 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d7699e-3cb3-47cb-a5c0-6cf0ba280695 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:20:49.189860 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 598/1213] 10.222.0.30 () {56 vars in 1143 bytes} [Tue Sep 6 04:20:49 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:20:49.195612 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222d7331-918e-4aab-8589-75b5ca2dd9c5 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:20:49.294998 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222d7331-918e-4aab-8589-75b5ca2dd9c5 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:20:49.327836 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de873cb5-f9f4-4b8f-b8b4-33083bfba11f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:20:49.328431 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de873cb5-f9f4-4b8f-b8b4-33083bfba11f None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:20:49.328705 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de873cb5-f9f4-4b8f-b8b4-33083bfba11f None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:20:49.328976 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de873cb5-f9f4-4b8f-b8b4-33083bfba11f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:20:49.358417 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:20:49.397404 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de873cb5-f9f4-4b8f-b8b4-33083bfba11f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:20:49.397404 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de873cb5-f9f4-4b8f-b8b4-33083bfba11f None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:20:49.491577 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 616/1214] 10.222.0.30 () {58 vars in 1282 bytes} [Tue Sep 6 04:20:49 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 06 04:20:49.497736 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d7699e-3cb3-47cb-a5c0-6cf0ba280695 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:20:49.516882 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:20:49.596602 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d7699e-3cb3-47cb-a5c0-6cf0ba280695 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:20:49.630074 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8efb6573-d997-4564-b816-16a6bd7b51f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:20:49.630695 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efb6573-d997-4564-b816-16a6bd7b51f5 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:20:49.630970 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efb6573-d997-4564-b816-16a6bd7b51f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:20:49.631251 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efb6573-d997-4564-b816-16a6bd7b51f5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:20:49.641060 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8efb6573-d997-4564-b816-16a6bd7b51f5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:20:49.643143 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8efb6573-d997-4564-b816-16a6bd7b51f5 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:20:49.657138 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-8efb6573-d997-4564-b816-16a6bd7b51f5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 04:20:49.657138 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 04:20:49.657138 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 04:20:49.657138 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 04:20:49.657138 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 04:20:49.657138 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 04:20:49.657138 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 04:20:49.657138 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 04:20:49.657138 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 04:20:49.657138 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 04:20:49.657138 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 04:20:49.657138 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 04:20:49.657138 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 04:20:49.657138 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 04:20:49.657138 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 04:20:49.657138 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 04:20:49.658340 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 04:20:49.658340 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 04:20:49.658340 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 04:20:49.658340 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 04:20:49.658340 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 04:20:49.658340 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 04:20:49.658340 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 04:20:49.658340 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 04:20:49.658340 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 04:20:49.658340 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 04:20:49.658340 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 04:20:49.658340 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 04:20:49.658340 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 04:20:49.658340 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 04:20:49.658340 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 04:20:49.658340 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 04:20:49.658340 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 04:20:49.659268 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 04:20:49.659268 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 04:20:49.659268 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 04:20:49.659268 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 599/1215] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Sep 6 04:20:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 04:20:49.664318 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de873cb5-f9f4-4b8f-b8b4-33083bfba11f None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:20:49.725480 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:20:49.764647 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de873cb5-f9f4-4b8f-b8b4-33083bfba11f None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:20:49.800064 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e3481e-ee93-4b10-8c6d-e1d332d15e8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:20:49.800672 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e3481e-ee93-4b10-8c6d-e1d332d15e8b None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:20:49.800950 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e3481e-ee93-4b10-8c6d-e1d332d15e8b None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:20:49.801221 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e3481e-ee93-4b10-8c6d-e1d332d15e8b None admin] PATH_INFO: `/v3/domains` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:20:49.801891 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e3481e-ee93-4b10-8c6d-e1d332d15e8b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:20:49.802191 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e3481e-ee93-4b10-8c6d-e1d332d15e8b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:20:49.803904 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e3481e-ee93-4b10-8c6d-e1d332d15e8b None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:20:49.815761 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 617/1216] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Sep 6 04:20:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:20:49.821713 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8efb6573-d997-4564-b816-16a6bd7b51f5 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:20:49.890608 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:20:49.920772 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8efb6573-d997-4564-b816-16a6bd7b51f5 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:20:49.954631 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e349cf9b-e74b-4dee-bf99-74335bad5ab3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:20:49.955426 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e349cf9b-e74b-4dee-bf99-74335bad5ab3 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:20:49.955710 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e349cf9b-e74b-4dee-bf99-74335bad5ab3 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:20:49.955985 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e349cf9b-e74b-4dee-bf99-74335bad5ab3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:20:49.962466 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e349cf9b-e74b-4dee-bf99-74335bad5ab3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:20:49.965548 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e349cf9b-e74b-4dee-bf99-74335bad5ab3 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:20:49.976884 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-e349cf9b-e74b-4dee-bf99-74335bad5ab3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 04:20:49.976884 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 04:20:49.976884 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 04:20:49.976884 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 04:20:49.976884 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 04:20:49.976884 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 04:20:49.976884 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 04:20:49.976884 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 04:20:49.976884 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 04:20:49.976884 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 04:20:49.976884 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 04:20:49.976884 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 04:20:49.976884 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 06 04:20:49.976884 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 06 04:20:49.976884 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 06 04:20:49.978149 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 06 04:20:49.978149 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 04:20:49.978149 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 04:20:49.978149 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 04:20:49.978149 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 04:20:49.978149 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 04:20:49.978149 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 04:20:49.978149 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 04:20:49.978149 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 04:20:49.978149 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 04:20:49.978149 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 04:20:49.978149 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 04:20:49.978149 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 04:20:49.978149 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 04:20:49.978149 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 04:20:49.978149 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 04:20:49.978149 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 04:20:49.979189 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 06 04:20:49.979189 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 06 04:20:49.979189 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 06 04:20:49.979189 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 06 04:20:49.979189 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 06 04:20:49.979189 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 06 04:20:49.979189 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 04:20:49.979189 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 04:20:49.979189 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 600/1217] 10.222.0.30 () {56 vars in 1111 bytes} [Tue Sep 6 04:20:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 04:20:49.984027 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e3481e-ee93-4b10-8c6d-e1d332d15e8b None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:20:50.082756 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e3481e-ee93-4b10-8c6d-e1d332d15e8b None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:20:50.105086 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:20:50.116533 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870c3923-7eef-473e-9669-4bd8b9d4ca50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:20:50.117140 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870c3923-7eef-473e-9669-4bd8b9d4ca50 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:20:50.117453 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870c3923-7eef-473e-9669-4bd8b9d4ca50 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:20:50.117729 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870c3923-7eef-473e-9669-4bd8b9d4ca50 None admin] PATH_INFO: `/v3/projects` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:20:50.118417 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870c3923-7eef-473e-9669-4bd8b9d4ca50 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:20:50.118722 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870c3923-7eef-473e-9669-4bd8b9d4ca50 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:20:50.120166 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870c3923-7eef-473e-9669-4bd8b9d4ca50 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:20:50.132544 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 618/1218] 10.222.0.30 () {56 vars in 1148 bytes} [Tue Sep 6 04:20:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:20:50.138635 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e349cf9b-e74b-4dee-bf99-74335bad5ab3 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:20:50.236454 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e349cf9b-e74b-4dee-bf99-74335bad5ab3 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:20:50.264933 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:20:50.271466 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4818d6-3c76-47d7-81d8-cfb1a854ffa4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:20:50.272118 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4818d6-3c76-47d7-81d8-cfb1a854ffa4 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:20:50.272397 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4818d6-3c76-47d7-81d8-cfb1a854ffa4 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:20:50.272667 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4818d6-3c76-47d7-81d8-cfb1a854ffa4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:20:50.273433 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4818d6-3c76-47d7-81d8-cfb1a854ffa4 None admin] RBAC: Adding query filter params (role.id=83cf0a194f8d45e792411e2d9afdfe50, scope.project.id=b707733ce65c42edbc1fd95ad0a1c06c, user.id=1d4f2e4dbd41441d84f03ddb30191e7a) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:20:50.276485 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4818d6-3c76-47d7-81d8-cfb1a854ffa4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:20:50.276485 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4818d6-3c76-47d7-81d8-cfb1a854ffa4 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:20:50.280337 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 601/1219] 10.222.0.30 () {56 vars in 1374 bytes} [Tue Sep 6 04:20:50 2022] GET /identity/v3/role_assignments?user.id=1d4f2e4dbd41441d84f03ddb30191e7a&scope.project.id=b707733ce65c42edbc1fd95ad0a1c06c&role.id=83cf0a194f8d45e792411e2d9afdfe50 => generated 259 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:20:52.227552 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774c3272-cf77-440a-af06-ca42a5ac6d5d None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:20:52.228103 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774c3272-cf77-440a-af06-ca42a5ac6d5d None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:20:52.228103 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774c3272-cf77-440a-af06-ca42a5ac6d5d None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:20:52.229472 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 619/1220] 10.222.0.30 () {54 vars in 927 bytes} [Tue Sep 6 04:20:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 04:20:52.234704 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb82e932-7e02-4691-9c26-71d6623cd1b6 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:20:52.235110 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb82e932-7e02-4691-9c26-71d6623cd1b6 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:20:52.235271 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb82e932-7e02-4691-9c26-71d6623cd1b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:20:52.255025 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-fb82e932-7e02-4691-9c26-71d6623cd1b6 None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:20:52.256216 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-fb82e932-7e02-4691-9c26-71d6623cd1b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:20:52.324460 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 602/1221] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Sep 6 04:20:52 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 04:20:52.606608 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774c3272-cf77-440a-af06-ca42a5ac6d5d None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:20:52.710791 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774c3272-cf77-440a-af06-ca42a5ac6d5d None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:20:52.745617 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f87ba29-5600-484f-bd75-6ad7741b4300 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:20:52.746250 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f87ba29-5600-484f-bd75-6ad7741b4300 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:20:52.746532 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f87ba29-5600-484f-bd75-6ad7741b4300 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:20:52.746804 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f87ba29-5600-484f-bd75-6ad7741b4300 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:20:52.757785 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f87ba29-5600-484f-bd75-6ad7741b4300 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 04:20:52.762676 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-1f87ba29-5600-484f-bd75-6ad7741b4300 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 06 04:20:52.762676 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 04:20:52.762676 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 04:20:52.762676 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 04:20:52.762676 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 04:20:52.762676 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 04:20:52.762676 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 04:20:52.762676 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 04:20:52.762676 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 04:20:52.762676 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 04:20:52.762676 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 04:20:52.762676 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 04:20:52.762676 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 04:20:52.762676 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 04:20:52.762676 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 04:20:52.762676 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 04:20:52.763793 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 04:20:52.763793 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 04:20:52.763793 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 04:20:52.763793 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 04:20:52.763793 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 04:20:52.763793 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 04:20:52.763793 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 04:20:52.763793 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 04:20:52.763793 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 04:20:52.763793 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 04:20:52.763793 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 04:20:52.763793 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 04:20:52.763793 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 04:20:52.763793 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 04:20:52.763793 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 04:20:52.763793 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 04:20:52.764824 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 04:20:52.764824 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 04:20:52.764824 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 04:20:52.764824 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 04:20:52.764824 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 04:20:52.764824 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 04:20:52.764824 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 04:20:52.764824 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 04:20:52.764824 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 04:20:52.764824 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 04:20:52.764824 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 06 04:20:52.764824 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 04:20:52.764824 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 620/1222] 10.222.0.30 () {56 vars in 1102 bytes} [Tue Sep 6 04:20:52 2022] GET /identity/v3/roles/service => generated 85 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 04:20:52.769878 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb82e932-7e02-4691-9c26-71d6623cd1b6 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:20:52.870549 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb82e932-7e02-4691-9c26-71d6623cd1b6 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:20:52.894228 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:20:52.904473 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f11e59-0478-4bcf-956d-f1c94a2b19be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:20:52.905086 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f11e59-0478-4bcf-956d-f1c94a2b19be None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:20:52.905396 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f11e59-0478-4bcf-956d-f1c94a2b19be None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:20:52.905673 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f11e59-0478-4bcf-956d-f1c94a2b19be None admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:20:52.906451 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f11e59-0478-4bcf-956d-f1c94a2b19be None admin] RBAC: Adding query filter params (name=service) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:20:52.906760 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f11e59-0478-4bcf-956d-f1c94a2b19be None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:20:52.908886 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f11e59-0478-4bcf-956d-f1c94a2b19be None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:20:52.916630 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 603/1223] 10.222.0.30 () {56 vars in 1103 bytes} [Tue Sep 6 04:20:52 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:20:52.922317 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f87ba29-5600-484f-bd75-6ad7741b4300 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:20:52.968783 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:20:53.024232 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f87ba29-5600-484f-bd75-6ad7741b4300 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:20:53.056732 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a77f992-2eab-4138-8103-210d18865bb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:20:53.057377 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a77f992-2eab-4138-8103-210d18865bb3 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:20:53.057662 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a77f992-2eab-4138-8103-210d18865bb3 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:20:53.057935 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a77f992-2eab-4138-8103-210d18865bb3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:20:53.068925 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a77f992-2eab-4138-8103-210d18865bb3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:20:53.070509 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a77f992-2eab-4138-8103-210d18865bb3 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:20:53.084801 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-3a77f992-2eab-4138-8103-210d18865bb3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 04:20:53.084801 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 04:20:53.084801 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 04:20:53.084801 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 04:20:53.084801 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 04:20:53.084801 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 04:20:53.084801 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 04:20:53.084801 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 04:20:53.084801 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 04:20:53.084801 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 04:20:53.084801 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 04:20:53.084801 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 04:20:53.084801 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 04:20:53.084801 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 04:20:53.084801 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 04:20:53.084801 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 04:20:53.086009 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 04:20:53.086009 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 04:20:53.086009 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 04:20:53.086009 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 04:20:53.086009 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 04:20:53.086009 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 04:20:53.086009 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 04:20:53.086009 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 04:20:53.086009 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 04:20:53.086009 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 04:20:53.086009 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 04:20:53.086009 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 04:20:53.086009 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 04:20:53.086009 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 04:20:53.086009 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 04:20:53.086009 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 04:20:53.086009 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 04:20:53.087085 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 04:20:53.087085 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 04:20:53.087085 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 04:20:53.087085 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 621/1224] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Sep 6 04:20:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 04:20:53.091745 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f11e59-0478-4bcf-956d-f1c94a2b19be None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:20:53.193076 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f11e59-0478-4bcf-956d-f1c94a2b19be None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:20:53.226981 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30669a7c-19ec-418d-8089-caa84122f856 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:20:53.227585 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30669a7c-19ec-418d-8089-caa84122f856 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:20:53.227867 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30669a7c-19ec-418d-8089-caa84122f856 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:20:53.228139 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30669a7c-19ec-418d-8089-caa84122f856 None admin] PATH_INFO: `/v3/domains` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:20:53.228775 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30669a7c-19ec-418d-8089-caa84122f856 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:20:53.229077 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30669a7c-19ec-418d-8089-caa84122f856 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:20:53.230478 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30669a7c-19ec-418d-8089-caa84122f856 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:20:53.242713 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 604/1225] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Sep 6 04:20:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:20:53.248648 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a77f992-2eab-4138-8103-210d18865bb3 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:20:53.343729 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:20:53.350419 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a77f992-2eab-4138-8103-210d18865bb3 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:20:53.384345 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86dbdcc3-64a3-4120-a9c3-d0ede096d1a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:20:53.384979 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86dbdcc3-64a3-4120-a9c3-d0ede096d1a2 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:20:53.385255 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86dbdcc3-64a3-4120-a9c3-d0ede096d1a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:20:53.385554 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86dbdcc3-64a3-4120-a9c3-d0ede096d1a2 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:20:53.393138 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86dbdcc3-64a3-4120-a9c3-d0ede096d1a2 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:20:53.395915 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86dbdcc3-64a3-4120-a9c3-d0ede096d1a2 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:20:53.405300 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-86dbdcc3-64a3-4120-a9c3-d0ede096d1a2 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 06 04:20:53.405300 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 04:20:53.405300 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 04:20:53.405300 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 04:20:53.405300 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 04:20:53.405300 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 04:20:53.405300 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 04:20:53.405300 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 04:20:53.405300 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 04:20:53.405300 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 04:20:53.405300 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 04:20:53.405300 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 04:20:53.405300 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 06 04:20:53.405300 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 06 04:20:53.405300 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 06 04:20:53.405300 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 06 04:20:53.406416 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 04:20:53.406416 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 04:20:53.406416 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 06 04:20:53.406416 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 04:20:53.406416 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 06 04:20:53.406416 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 04:20:53.406416 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 04:20:53.406416 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 04:20:53.406416 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 04:20:53.406416 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 04:20:53.406416 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 04:20:53.406416 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 04:20:53.406416 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 04:20:53.406416 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 04:20:53.406416 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 04:20:53.406416 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 04:20:53.406416 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 04:20:53.406416 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 04:20:53.407779 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 04:20:53.407779 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 04:20:53.407779 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 06 04:20:53.407779 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 06 04:20:53.407779 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 06 04:20:53.407779 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 06 04:20:53.407779 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 06 04:20:53.407779 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 06 04:20:53.407779 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 06 04:20:53.407779 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 04:20:53.407779 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 622/1226] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Sep 6 04:20:53 2022] GET /identity/v3/users/placement => generated 87 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 04:20:53.412488 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30669a7c-19ec-418d-8089-caa84122f856 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:20:53.459159 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:20:53.513007 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30669a7c-19ec-418d-8089-caa84122f856 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:20:53.545945 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32bf7f87-10a9-4234-b7a3-f46e725152b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:20:53.546584 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bf7f87-10a9-4234-b7a3-f46e725152b5 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:20:53.546864 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bf7f87-10a9-4234-b7a3-f46e725152b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:20:53.547136 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bf7f87-10a9-4234-b7a3-f46e725152b5 None admin] PATH_INFO: `/v3/users` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:20:53.548475 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32bf7f87-10a9-4234-b7a3-f46e725152b5 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:20:53.549014 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32bf7f87-10a9-4234-b7a3-f46e725152b5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:20:53.550479 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32bf7f87-10a9-4234-b7a3-f46e725152b5 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:20:53.569297 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 605/1227] 10.222.0.30 () {56 vars in 1143 bytes} [Tue Sep 6 04:20:53 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:20:53.575079 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86dbdcc3-64a3-4120-a9c3-d0ede096d1a2 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:20:53.675140 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86dbdcc3-64a3-4120-a9c3-d0ede096d1a2 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:20:53.712128 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439433f9-3925-4df6-bce7-e89a68362666 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:20:53.712683 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439433f9-3925-4df6-bce7-e89a68362666 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:20:53.712962 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439433f9-3925-4df6-bce7-e89a68362666 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:20:53.713237 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439433f9-3925-4df6-bce7-e89a68362666 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:20:53.724556 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-439433f9-3925-4df6-bce7-e89a68362666 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:20:53.726033 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-439433f9-3925-4df6-bce7-e89a68362666 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:20:53.740105 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-439433f9-3925-4df6-bce7-e89a68362666 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 04:20:53.740105 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 04:20:53.740105 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 04:20:53.740105 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 04:20:53.740105 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 04:20:53.740105 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 04:20:53.740105 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 04:20:53.740105 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 04:20:53.740105 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 04:20:53.740105 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 04:20:53.740105 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 04:20:53.740105 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 04:20:53.740105 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 04:20:53.740105 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 04:20:53.740105 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 04:20:53.740105 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 04:20:53.741118 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 04:20:53.741118 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 04:20:53.741118 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 04:20:53.741118 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 04:20:53.741118 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 04:20:53.741118 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 04:20:53.741118 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 04:20:53.741118 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 04:20:53.741118 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 04:20:53.741118 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 04:20:53.741118 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 04:20:53.741118 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 04:20:53.741118 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 04:20:53.741118 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 04:20:53.741118 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 04:20:53.741118 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 04:20:53.741118 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 04:20:53.742474 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 04:20:53.742474 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 04:20:53.742474 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 04:20:53.742474 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 623/1228] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Sep 6 04:20:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 04:20:53.747100 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32bf7f87-10a9-4234-b7a3-f46e725152b5 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:20:53.846702 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32bf7f87-10a9-4234-b7a3-f46e725152b5 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:20:53.848998 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:20:53.881771 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d574a73-c361-41a1-b56d-6f7bc646f5a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:20:53.882376 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d574a73-c361-41a1-b56d-6f7bc646f5a3 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:20:53.882654 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d574a73-c361-41a1-b56d-6f7bc646f5a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:20:53.882926 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d574a73-c361-41a1-b56d-6f7bc646f5a3 None admin] PATH_INFO: `/v3/domains` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:20:53.883567 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d574a73-c361-41a1-b56d-6f7bc646f5a3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:20:53.883870 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d574a73-c361-41a1-b56d-6f7bc646f5a3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:20:53.885228 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d574a73-c361-41a1-b56d-6f7bc646f5a3 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:20:53.897146 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 606/1229] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Sep 6 04:20:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:20:53.903084 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439433f9-3925-4df6-bce7-e89a68362666 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:20:53.913238 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:20:54.003700 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439433f9-3925-4df6-bce7-e89a68362666 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:20:54.037032 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8ada32-202c-45a9-9304-7576546b1f2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:20:54.037710 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8ada32-202c-45a9-9304-7576546b1f2b None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:20:54.037986 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8ada32-202c-45a9-9304-7576546b1f2b None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:20:54.038255 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8ada32-202c-45a9-9304-7576546b1f2b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:20:54.045273 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8ada32-202c-45a9-9304-7576546b1f2b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:20:54.046778 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8ada32-202c-45a9-9304-7576546b1f2b None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:20:54.060865 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-ba8ada32-202c-45a9-9304-7576546b1f2b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 04:20:54.060865 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 04:20:54.060865 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 04:20:54.060865 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 04:20:54.060865 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 04:20:54.060865 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 04:20:54.060865 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 04:20:54.060865 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 04:20:54.060865 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 04:20:54.060865 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 04:20:54.060865 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 04:20:54.060865 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 04:20:54.060865 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 06 04:20:54.060865 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 06 04:20:54.060865 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 06 04:20:54.062554 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 06 04:20:54.062554 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 04:20:54.062554 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 04:20:54.062554 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 04:20:54.062554 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 04:20:54.062554 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 04:20:54.062554 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 04:20:54.062554 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 04:20:54.062554 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 04:20:54.062554 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 04:20:54.062554 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 04:20:54.062554 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 04:20:54.062554 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 04:20:54.062554 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 04:20:54.062554 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 04:20:54.062554 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 04:20:54.062554 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 04:20:54.064677 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 06 04:20:54.064677 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 06 04:20:54.064677 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 06 04:20:54.064677 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 06 04:20:54.064677 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 06 04:20:54.064677 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 06 04:20:54.064677 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 04:20:54.064677 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 04:20:54.064677 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 624/1230] 10.222.0.30 () {56 vars in 1111 bytes} [Tue Sep 6 04:20:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 04:20:54.070588 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d574a73-c361-41a1-b56d-6f7bc646f5a3 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:20:54.172655 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d574a73-c361-41a1-b56d-6f7bc646f5a3 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:20:54.207721 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36e9195-4b48-4aa6-9f3b-20db70542416 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:20:54.208331 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36e9195-4b48-4aa6-9f3b-20db70542416 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:20:54.208609 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36e9195-4b48-4aa6-9f3b-20db70542416 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:20:54.208876 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36e9195-4b48-4aa6-9f3b-20db70542416 None admin] PATH_INFO: `/v3/projects` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:20:54.209603 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d36e9195-4b48-4aa6-9f3b-20db70542416 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:20:54.209906 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d36e9195-4b48-4aa6-9f3b-20db70542416 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:20:54.211327 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d36e9195-4b48-4aa6-9f3b-20db70542416 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:20:54.224100 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 607/1231] 10.222.0.30 () {56 vars in 1148 bytes} [Tue Sep 6 04:20:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:20:54.230188 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8ada32-202c-45a9-9304-7576546b1f2b None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:20:54.295388 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:20:54.330250 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8ada32-202c-45a9-9304-7576546b1f2b None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:20:54.367089 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4e94a7-d07a-4ee3-bb91-b5edb37b3d1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:20:54.367828 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4e94a7-d07a-4ee3-bb91-b5edb37b3d1d None admin] REQUEST_METHOD: `PUT` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:20:54.368110 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4e94a7-d07a-4ee3-bb91-b5edb37b3d1d None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:20:54.368378 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4e94a7-d07a-4ee3-bb91-b5edb37b3d1d None admin] PATH_INFO: `/v3/projects/b707733ce65c42edbc1fd95ad0a1c06c/users/1d4f2e4dbd41441d84f03ddb30191e7a/roles/83cf0a194f8d45e792411e2d9afdfe50` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:20:54.384043 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4e94a7-d07a-4ee3-bb91-b5edb37b3d1d None admin] RBAC: Authorizing `identity:create_grant(project_id=b707733ce65c42edbc1fd95ad0a1c06c, user_id=1d4f2e4dbd41441d84f03ddb30191e7a, role_id=83cf0a194f8d45e792411e2d9afdfe50)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:20:54.385105 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4e94a7-d07a-4ee3-bb91-b5edb37b3d1d None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:20:54.443393 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 625/1232] 10.222.0.30 () {58 vars in 1439 bytes} [Tue Sep 6 04:20:54 2022] PUT /identity/v3/projects/b707733ce65c42edbc1fd95ad0a1c06c/users/1d4f2e4dbd41441d84f03ddb30191e7a/roles/83cf0a194f8d45e792411e2d9afdfe50 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 04:20:56.404123 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7756d0a8-7f93-4992-9c05-eb898f18f818 None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:20:56.404593 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7756d0a8-7f93-4992-9c05-eb898f18f818 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:20:56.404696 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7756d0a8-7f93-4992-9c05-eb898f18f818 None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:20:56.406039 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 608/1233] 10.222.0.30 () {54 vars in 927 bytes} [Tue Sep 6 04:20:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 04:20:56.411983 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44736d0-f774-4963-9ea1-116ef659e8ba None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:20:56.412579 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44736d0-f774-4963-9ea1-116ef659e8ba None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:20:56.412579 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44736d0-f774-4963-9ea1-116ef659e8ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:20:56.431232 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-b44736d0-f774-4963-9ea1-116ef659e8ba None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:20:56.432463 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-b44736d0-f774-4963-9ea1-116ef659e8ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:20:56.511555 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 626/1234] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Sep 6 04:20:56 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 04:20:56.789891 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7756d0a8-7f93-4992-9c05-eb898f18f818 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:20:56.815800 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:20:56.890331 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7756d0a8-7f93-4992-9c05-eb898f18f818 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:20:56.924413 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afefa36d-0f7a-4595-91f8-dedbc7d4f047 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:20:56.925037 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afefa36d-0f7a-4595-91f8-dedbc7d4f047 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:20:56.925336 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afefa36d-0f7a-4595-91f8-dedbc7d4f047 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:20:56.925616 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afefa36d-0f7a-4595-91f8-dedbc7d4f047 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:20:56.935444 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afefa36d-0f7a-4595-91f8-dedbc7d4f047 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 04:20:56.941021 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-afefa36d-0f7a-4595-91f8-dedbc7d4f047 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 06 04:20:56.941021 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 04:20:56.941021 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 04:20:56.941021 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 04:20:56.941021 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 04:20:56.941021 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 04:20:56.941021 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 04:20:56.941021 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 04:20:56.941021 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 04:20:56.941021 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 04:20:56.941021 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 04:20:56.941021 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 04:20:56.941021 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 04:20:56.941021 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 04:20:56.941021 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 04:20:56.941021 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 04:20:56.942131 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 04:20:56.942131 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 04:20:56.942131 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 04:20:56.942131 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 04:20:56.942131 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 04:20:56.942131 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 04:20:56.942131 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 04:20:56.942131 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 04:20:56.942131 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 04:20:56.942131 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 04:20:56.942131 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 04:20:56.942131 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 04:20:56.942131 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 04:20:56.942131 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 04:20:56.942131 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 04:20:56.942131 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 04:20:56.943438 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 04:20:56.943438 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 04:20:56.943438 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 04:20:56.943438 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 04:20:56.943438 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 04:20:56.943438 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 04:20:56.943438 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 04:20:56.943438 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 04:20:56.943438 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 04:20:56.943438 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 04:20:56.943438 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 06 04:20:56.943438 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 04:20:56.943438 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 609/1235] 10.222.0.30 () {56 vars in 1102 bytes} [Tue Sep 6 04:20:56 2022] GET /identity/v3/roles/service => generated 85 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 04:20:56.948162 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44736d0-f774-4963-9ea1-116ef659e8ba None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:20:56.993516 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:20:57.044486 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44736d0-f774-4963-9ea1-116ef659e8ba None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:20:57.083547 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7150c8b-8e83-46a9-8875-7bb1f1271db2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:20:57.084161 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7150c8b-8e83-46a9-8875-7bb1f1271db2 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:20:57.084437 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7150c8b-8e83-46a9-8875-7bb1f1271db2 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:20:57.084703 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7150c8b-8e83-46a9-8875-7bb1f1271db2 None admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:20:57.085398 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7150c8b-8e83-46a9-8875-7bb1f1271db2 None admin] RBAC: Adding query filter params (name=service) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:20:57.085698 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7150c8b-8e83-46a9-8875-7bb1f1271db2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:20:57.087303 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7150c8b-8e83-46a9-8875-7bb1f1271db2 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:20:57.095994 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 627/1236] 10.222.0.30 () {56 vars in 1103 bytes} [Tue Sep 6 04:20:56 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:20:57.102059 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afefa36d-0f7a-4595-91f8-dedbc7d4f047 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:20:57.186863 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:20:57.202184 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afefa36d-0f7a-4595-91f8-dedbc7d4f047 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:20:57.235993 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb816b0-4b73-4f6d-85df-23fe6954f474 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:20:57.236620 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb816b0-4b73-4f6d-85df-23fe6954f474 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:20:57.236902 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb816b0-4b73-4f6d-85df-23fe6954f474 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:20:57.237175 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb816b0-4b73-4f6d-85df-23fe6954f474 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:20:57.246804 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb816b0-4b73-4f6d-85df-23fe6954f474 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:20:57.248231 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb816b0-4b73-4f6d-85df-23fe6954f474 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:20:57.262879 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-4fb816b0-4b73-4f6d-85df-23fe6954f474 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 04:20:57.262879 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 04:20:57.262879 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 04:20:57.262879 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 04:20:57.262879 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 04:20:57.262879 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 04:20:57.262879 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 04:20:57.262879 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 04:20:57.262879 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 04:20:57.262879 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 04:20:57.262879 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 04:20:57.262879 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 04:20:57.262879 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 04:20:57.262879 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 04:20:57.262879 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 04:20:57.262879 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 04:20:57.263853 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 04:20:57.263853 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 04:20:57.263853 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 04:20:57.263853 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 04:20:57.263853 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 04:20:57.263853 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 04:20:57.263853 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 04:20:57.263853 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 04:20:57.263853 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 04:20:57.263853 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 04:20:57.263853 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 04:20:57.263853 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 04:20:57.263853 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 04:20:57.263853 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 04:20:57.263853 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 04:20:57.263853 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 04:20:57.263853 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 04:20:57.264807 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 04:20:57.264807 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 04:20:57.264807 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 04:20:57.264807 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 610/1237] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Sep 6 04:20:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 04:20:57.270880 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7150c8b-8e83-46a9-8875-7bb1f1271db2 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:20:57.371041 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7150c8b-8e83-46a9-8875-7bb1f1271db2 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:20:57.381064 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:20:57.409530 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ef7c0a-cdee-4b6e-b497-86207883d481 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:20:57.410142 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ef7c0a-cdee-4b6e-b497-86207883d481 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:20:57.410421 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ef7c0a-cdee-4b6e-b497-86207883d481 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:20:57.410691 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ef7c0a-cdee-4b6e-b497-86207883d481 None admin] PATH_INFO: `/v3/domains` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:20:57.411331 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ef7c0a-cdee-4b6e-b497-86207883d481 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:20:57.411629 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ef7c0a-cdee-4b6e-b497-86207883d481 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:20:57.412995 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ef7c0a-cdee-4b6e-b497-86207883d481 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:20:57.425207 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 628/1238] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Sep 6 04:20:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:20:57.431024 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb816b0-4b73-4f6d-85df-23fe6954f474 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:20:57.533900 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb816b0-4b73-4f6d-85df-23fe6954f474 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:20:57.567435 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04567f2-a7bf-463c-a054-78adc28a6f48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:20:57.568064 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04567f2-a7bf-463c-a054-78adc28a6f48 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:20:57.568345 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04567f2-a7bf-463c-a054-78adc28a6f48 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:20:57.568622 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04567f2-a7bf-463c-a054-78adc28a6f48 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:20:57.571330 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:20:57.576242 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04567f2-a7bf-463c-a054-78adc28a6f48 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:20:57.577702 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04567f2-a7bf-463c-a054-78adc28a6f48 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:20:57.588736 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-e04567f2-a7bf-463c-a054-78adc28a6f48 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 06 04:20:57.588736 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 04:20:57.588736 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 04:20:57.588736 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 04:20:57.588736 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 04:20:57.588736 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 04:20:57.588736 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 04:20:57.588736 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 04:20:57.588736 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 04:20:57.588736 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 04:20:57.588736 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 04:20:57.588736 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 04:20:57.588736 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 06 04:20:57.588736 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 06 04:20:57.588736 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 06 04:20:57.588736 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 06 04:20:57.590668 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 04:20:57.590668 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 04:20:57.590668 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 06 04:20:57.590668 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 04:20:57.590668 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 06 04:20:57.590668 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 04:20:57.590668 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 04:20:57.590668 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 04:20:57.590668 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 04:20:57.590668 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 04:20:57.590668 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 04:20:57.590668 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 04:20:57.590668 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 04:20:57.590668 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 04:20:57.590668 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 04:20:57.590668 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 04:20:57.590668 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 04:20:57.590668 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 04:20:57.591857 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 04:20:57.591857 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 04:20:57.591857 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 06 04:20:57.591857 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 06 04:20:57.591857 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 06 04:20:57.591857 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 06 04:20:57.591857 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 06 04:20:57.591857 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 06 04:20:57.591857 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 06 04:20:57.591857 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 04:20:57.591857 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 611/1239] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Sep 6 04:20:57 2022] GET /identity/v3/users/placement => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 04:20:57.596543 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ef7c0a-cdee-4b6e-b497-86207883d481 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:20:57.701732 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ef7c0a-cdee-4b6e-b497-86207883d481 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:20:57.736626 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5aca8d-72bf-4358-a4ea-7b8591ccc562 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:20:57.737231 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5aca8d-72bf-4358-a4ea-7b8591ccc562 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:20:57.737542 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5aca8d-72bf-4358-a4ea-7b8591ccc562 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:20:57.737815 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5aca8d-72bf-4358-a4ea-7b8591ccc562 None admin] PATH_INFO: `/v3/users` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:20:57.738538 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc5aca8d-72bf-4358-a4ea-7b8591ccc562 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:20:57.738839 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc5aca8d-72bf-4358-a4ea-7b8591ccc562 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:20:57.740349 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc5aca8d-72bf-4358-a4ea-7b8591ccc562 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:20:57.758063 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 629/1240] 10.222.0.30 () {56 vars in 1143 bytes} [Tue Sep 6 04:20:57 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 04:20:57.763934 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04567f2-a7bf-463c-a054-78adc28a6f48 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:20:57.865374 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04567f2-a7bf-463c-a054-78adc28a6f48 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:20:57.904466 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ad9374-9384-4521-b6eb-f716b02ed430 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:20:57.905068 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ad9374-9384-4521-b6eb-f716b02ed430 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:20:57.905367 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ad9374-9384-4521-b6eb-f716b02ed430 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:20:57.905639 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ad9374-9384-4521-b6eb-f716b02ed430 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:20:57.964370 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:20:57.970807 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ad9374-9384-4521-b6eb-f716b02ed430 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:20:57.972236 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ad9374-9384-4521-b6eb-f716b02ed430 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:20:58.068120 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 612/1241] 10.222.0.30 () {58 vars in 1282 bytes} [Tue Sep 6 04:20:57 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 06 04:20:58.074007 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5aca8d-72bf-4358-a4ea-7b8591ccc562 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:20:58.100993 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:20:58.171897 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5aca8d-72bf-4358-a4ea-7b8591ccc562 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:20:58.206749 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b7ce5d-e0c6-48d4-a2b8-8f7d973c603d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:20:58.207388 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b7ce5d-e0c6-48d4-a2b8-8f7d973c603d None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:20:58.207658 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b7ce5d-e0c6-48d4-a2b8-8f7d973c603d None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:20:58.207929 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b7ce5d-e0c6-48d4-a2b8-8f7d973c603d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:20:58.217660 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b7ce5d-e0c6-48d4-a2b8-8f7d973c603d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:20:58.219050 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b7ce5d-e0c6-48d4-a2b8-8f7d973c603d None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:20:58.233777 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-27b7ce5d-e0c6-48d4-a2b8-8f7d973c603d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 04:20:58.233777 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 04:20:58.233777 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 04:20:58.233777 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 04:20:58.233777 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 04:20:58.233777 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 04:20:58.233777 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 04:20:58.233777 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 04:20:58.233777 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 04:20:58.233777 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 04:20:58.233777 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 04:20:58.233777 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 04:20:58.233777 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 04:20:58.233777 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 04:20:58.233777 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 04:20:58.233777 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 04:20:58.234982 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 04:20:58.234982 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 04:20:58.234982 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 04:20:58.234982 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 04:20:58.234982 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 04:20:58.234982 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 04:20:58.234982 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 04:20:58.234982 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 04:20:58.234982 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 04:20:58.234982 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 04:20:58.234982 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 04:20:58.234982 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 04:20:58.234982 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 04:20:58.234982 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 04:20:58.234982 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 04:20:58.234982 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 04:20:58.234982 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 04:20:58.236141 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 04:20:58.236141 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 04:20:58.236141 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 04:20:58.236141 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 630/1242] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Sep 6 04:20:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 04:20:58.241440 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ad9374-9384-4521-b6eb-f716b02ed430 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:20:58.332610 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:20:58.338200 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ad9374-9384-4521-b6eb-f716b02ed430 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:20:58.372628 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81bcd91-0d3c-4c41-adb4-d40f8a795e89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:20:58.373249 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81bcd91-0d3c-4c41-adb4-d40f8a795e89 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:20:58.373563 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81bcd91-0d3c-4c41-adb4-d40f8a795e89 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:20:58.373839 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81bcd91-0d3c-4c41-adb4-d40f8a795e89 None admin] PATH_INFO: `/v3/domains` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:20:58.374545 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a81bcd91-0d3c-4c41-adb4-d40f8a795e89 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:20:58.374806 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a81bcd91-0d3c-4c41-adb4-d40f8a795e89 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:20:58.376258 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a81bcd91-0d3c-4c41-adb4-d40f8a795e89 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:20:58.388817 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 613/1243] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Sep 6 04:20:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:20:58.394887 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b7ce5d-e0c6-48d4-a2b8-8f7d973c603d None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:20:58.474331 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:20:58.495796 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b7ce5d-e0c6-48d4-a2b8-8f7d973c603d None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:20:58.528200 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b2a129-ff80-42ff-ad29-bf9e8c242543 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:20:58.528817 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b2a129-ff80-42ff-ad29-bf9e8c242543 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:20:58.529095 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b2a129-ff80-42ff-ad29-bf9e8c242543 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:20:58.529390 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b2a129-ff80-42ff-ad29-bf9e8c242543 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:20:58.536358 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b2a129-ff80-42ff-ad29-bf9e8c242543 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:20:58.537850 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b2a129-ff80-42ff-ad29-bf9e8c242543 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:20:58.549877 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-32b2a129-ff80-42ff-ad29-bf9e8c242543 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 04:20:58.549877 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 04:20:58.549877 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 04:20:58.549877 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 04:20:58.549877 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 04:20:58.549877 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 04:20:58.549877 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 04:20:58.549877 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 04:20:58.549877 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 04:20:58.549877 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 04:20:58.549877 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 04:20:58.549877 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 04:20:58.549877 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 06 04:20:58.549877 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 06 04:20:58.549877 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 06 04:20:58.551222 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 06 04:20:58.551222 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 04:20:58.551222 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 04:20:58.551222 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 04:20:58.551222 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 04:20:58.551222 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 04:20:58.551222 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 04:20:58.551222 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 04:20:58.551222 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 04:20:58.551222 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 04:20:58.551222 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 04:20:58.551222 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 04:20:58.551222 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 04:20:58.551222 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 04:20:58.551222 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 04:20:58.551222 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 04:20:58.551222 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 04:20:58.552399 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 06 04:20:58.552399 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 06 04:20:58.552399 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 06 04:20:58.552399 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 06 04:20:58.552399 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 06 04:20:58.552399 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 06 04:20:58.552399 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 04:20:58.552399 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 04:20:58.552399 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 631/1244] 10.222.0.30 () {56 vars in 1111 bytes} [Tue Sep 6 04:20:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 04:20:58.559187 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81bcd91-0d3c-4c41-adb4-d40f8a795e89 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:20:58.662186 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81bcd91-0d3c-4c41-adb4-d40f8a795e89 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:20:58.695664 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1721f44-48b2-4311-a20c-6d9d849b5ecc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:20:58.696267 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1721f44-48b2-4311-a20c-6d9d849b5ecc None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:20:58.696545 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1721f44-48b2-4311-a20c-6d9d849b5ecc None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:20:58.696815 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1721f44-48b2-4311-a20c-6d9d849b5ecc None admin] PATH_INFO: `/v3/projects` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:20:58.697544 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1721f44-48b2-4311-a20c-6d9d849b5ecc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:20:58.697847 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1721f44-48b2-4311-a20c-6d9d849b5ecc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:20:58.699267 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1721f44-48b2-4311-a20c-6d9d849b5ecc None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:20:58.713207 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 614/1245] 10.222.0.30 () {56 vars in 1148 bytes} [Tue Sep 6 04:20:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:20:58.720165 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b2a129-ff80-42ff-ad29-bf9e8c242543 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:20:58.820921 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b2a129-ff80-42ff-ad29-bf9e8c242543 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:20:58.852868 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81a8101-661b-49b5-a38b-a61af2d96919 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:20:58.853490 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81a8101-661b-49b5-a38b-a61af2d96919 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:20:58.855719 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81a8101-661b-49b5-a38b-a61af2d96919 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:20:58.855719 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81a8101-661b-49b5-a38b-a61af2d96919 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:20:58.855719 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d81a8101-661b-49b5-a38b-a61af2d96919 None admin] RBAC: Adding query filter params (role.id=83cf0a194f8d45e792411e2d9afdfe50, scope.project.id=b707733ce65c42edbc1fd95ad0a1c06c, user.id=1d4f2e4dbd41441d84f03ddb30191e7a) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:20:58.855719 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d81a8101-661b-49b5-a38b-a61af2d96919 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:20:58.856399 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d81a8101-661b-49b5-a38b-a61af2d96919 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:20:58.862551 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 632/1246] 10.222.0.30 () {56 vars in 1374 bytes} [Tue Sep 6 04:20:58 2022] GET /identity/v3/role_assignments?user.id=1d4f2e4dbd41441d84f03ddb30191e7a&scope.project.id=b707733ce65c42edbc1fd95ad0a1c06c&role.id=83cf0a194f8d45e792411e2d9afdfe50 => generated 608 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:21:00.822000 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ba69c8-7130-451c-a345-baf37813ce6e None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:21:00.822528 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ba69c8-7130-451c-a345-baf37813ce6e None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:21:00.822528 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ba69c8-7130-451c-a345-baf37813ce6e None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:21:00.823870 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 615/1247] 10.222.0.30 () {54 vars in 927 bytes} [Tue Sep 6 04:21:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 04:21:00.829901 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d17c9f4-7e12-4283-9f82-fc997f0bb44e None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:21:00.830352 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d17c9f4-7e12-4283-9f82-fc997f0bb44e None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:21:00.830614 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d17c9f4-7e12-4283-9f82-fc997f0bb44e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:21:00.836945 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:21:00.849809 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-8d17c9f4-7e12-4283-9f82-fc997f0bb44e None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:21:00.850994 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-8d17c9f4-7e12-4283-9f82-fc997f0bb44e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:21:00.917956 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 633/1248] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Sep 6 04:21:00 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 04:21:01.190150 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ba69c8-7130-451c-a345-baf37813ce6e None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:21:01.208478 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:21:01.290106 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ba69c8-7130-451c-a345-baf37813ce6e None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:21:01.324877 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d15150-db15-4d96-9412-a78569717554 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:21:01.325530 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d15150-db15-4d96-9412-a78569717554 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:21:01.325814 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d15150-db15-4d96-9412-a78569717554 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:21:01.326089 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d15150-db15-4d96-9412-a78569717554 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:21:01.336131 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d15150-db15-4d96-9412-a78569717554 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 04:21:01.340895 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-24d15150-db15-4d96-9412-a78569717554 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 04:21:01.340895 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 04:21:01.340895 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 04:21:01.340895 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 04:21:01.340895 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 04:21:01.340895 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 04:21:01.340895 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 04:21:01.340895 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 04:21:01.340895 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 04:21:01.340895 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 04:21:01.340895 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 04:21:01.340895 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 04:21:01.340895 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 04:21:01.340895 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 04:21:01.340895 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 04:21:01.340895 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 04:21:01.342025 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 04:21:01.342025 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 04:21:01.342025 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 04:21:01.342025 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 04:21:01.342025 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 04:21:01.342025 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 04:21:01.342025 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 04:21:01.342025 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 04:21:01.342025 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 04:21:01.342025 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 04:21:01.342025 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 04:21:01.342025 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 04:21:01.342025 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 04:21:01.342025 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 04:21:01.342025 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 04:21:01.342025 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 04:21:01.343221 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 04:21:01.343221 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 04:21:01.343221 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 04:21:01.343221 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 04:21:01.343221 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 04:21:01.343221 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 04:21:01.343221 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 04:21:01.343221 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 04:21:01.343221 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 04:21:01.343221 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 04:21:01.343221 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 04:21:01.343221 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 04:21:01.343221 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 616/1249] 10.222.0.30 () {56 vars in 1096 bytes} [Tue Sep 6 04:21:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 04:21:01.348512 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d17c9f4-7e12-4283-9f82-fc997f0bb44e None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:21:01.447639 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d17c9f4-7e12-4283-9f82-fc997f0bb44e None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:21:01.477980 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:21:01.482767 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734266dd-a863-4086-9502-b819af69c95e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:21:01.483382 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734266dd-a863-4086-9502-b819af69c95e None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:21:01.483658 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734266dd-a863-4086-9502-b819af69c95e None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:21:01.483926 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734266dd-a863-4086-9502-b819af69c95e None admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:21:01.484607 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-734266dd-a863-4086-9502-b819af69c95e None admin] RBAC: Adding query filter params (name=admin) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:21:01.484905 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-734266dd-a863-4086-9502-b819af69c95e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:21:01.486963 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-734266dd-a863-4086-9502-b819af69c95e None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:21:01.494773 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 634/1250] 10.222.0.30 () {56 vars in 1099 bytes} [Tue Sep 6 04:21:01 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:21:01.500555 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d15150-db15-4d96-9412-a78569717554 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:21:01.576334 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:21:01.600370 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d15150-db15-4d96-9412-a78569717554 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:21:01.634659 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c6bc5b-13a4-4405-9981-8fa1debd2a2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:21:01.635289 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c6bc5b-13a4-4405-9981-8fa1debd2a2e None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:21:01.635570 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c6bc5b-13a4-4405-9981-8fa1debd2a2e None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:21:01.635841 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c6bc5b-13a4-4405-9981-8fa1debd2a2e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:21:01.646129 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c6bc5b-13a4-4405-9981-8fa1debd2a2e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:21:01.647567 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c6bc5b-13a4-4405-9981-8fa1debd2a2e None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:21:01.661867 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-30c6bc5b-13a4-4405-9981-8fa1debd2a2e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 04:21:01.661867 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 04:21:01.661867 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 04:21:01.661867 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 04:21:01.661867 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 04:21:01.661867 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 04:21:01.661867 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 04:21:01.661867 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 04:21:01.661867 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 04:21:01.661867 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 04:21:01.661867 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 04:21:01.661867 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 04:21:01.661867 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 04:21:01.661867 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 04:21:01.661867 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 04:21:01.661867 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 04:21:01.663000 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 04:21:01.663000 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 04:21:01.663000 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 04:21:01.663000 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 04:21:01.663000 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 04:21:01.663000 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 04:21:01.663000 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 04:21:01.663000 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 04:21:01.663000 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 04:21:01.663000 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 04:21:01.663000 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 04:21:01.663000 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 04:21:01.663000 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 04:21:01.663000 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 04:21:01.663000 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 04:21:01.663000 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 04:21:01.663000 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 04:21:01.663900 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 04:21:01.663900 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 04:21:01.663900 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 04:21:01.663900 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 617/1251] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Sep 6 04:21:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 04:21:01.668865 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734266dd-a863-4086-9502-b819af69c95e None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:21:01.768106 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734266dd-a863-4086-9502-b819af69c95e None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:21:01.801494 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997db285-ab25-4e64-9c53-ab61fec08ffa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:21:01.802181 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997db285-ab25-4e64-9c53-ab61fec08ffa None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:21:01.802462 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997db285-ab25-4e64-9c53-ab61fec08ffa None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:21:01.802732 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997db285-ab25-4e64-9c53-ab61fec08ffa None admin] PATH_INFO: `/v3/domains` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:21:01.803370 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997db285-ab25-4e64-9c53-ab61fec08ffa None admin] RBAC: Adding query filter params (name=Default) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:21:01.803670 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997db285-ab25-4e64-9c53-ab61fec08ffa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:21:01.805030 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997db285-ab25-4e64-9c53-ab61fec08ffa None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:21:01.817203 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 635/1252] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Sep 6 04:21:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:21:01.822936 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c6bc5b-13a4-4405-9981-8fa1debd2a2e None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:21:01.927932 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c6bc5b-13a4-4405-9981-8fa1debd2a2e None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:21:01.954159 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:21:01.961854 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6713c2-3c04-48c6-a6e0-481e708c04ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:21:01.962488 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6713c2-3c04-48c6-a6e0-481e708c04ed None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:21:01.962799 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6713c2-3c04-48c6-a6e0-481e708c04ed None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:21:01.963045 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6713c2-3c04-48c6-a6e0-481e708c04ed None admin] PATH_INFO: `/v3/users/placement` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:21:01.971139 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e6713c2-3c04-48c6-a6e0-481e708c04ed None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:21:01.972571 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e6713c2-3c04-48c6-a6e0-481e708c04ed None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:21:01.983485 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-4e6713c2-3c04-48c6-a6e0-481e708c04ed None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 06 04:21:01.983485 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 04:21:01.983485 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 04:21:01.983485 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 04:21:01.983485 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 04:21:01.983485 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 04:21:01.983485 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 04:21:01.983485 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 04:21:01.983485 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 04:21:01.983485 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 04:21:01.983485 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 04:21:01.983485 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 04:21:01.983485 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 06 04:21:01.983485 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 06 04:21:01.983485 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 06 04:21:01.983485 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 06 04:21:01.984615 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 04:21:01.984615 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 04:21:01.984615 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 06 04:21:01.984615 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 04:21:01.984615 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 06 04:21:01.984615 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 04:21:01.984615 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 04:21:01.984615 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 04:21:01.984615 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 04:21:01.984615 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 04:21:01.984615 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 04:21:01.984615 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 04:21:01.984615 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 04:21:01.984615 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 04:21:01.984615 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 04:21:01.984615 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 04:21:01.984615 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 04:21:01.984615 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 04:21:01.986255 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 04:21:01.986255 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 04:21:01.986255 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 06 04:21:01.986255 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 06 04:21:01.986255 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 06 04:21:01.986255 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 06 04:21:01.986255 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 06 04:21:01.986255 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 06 04:21:01.986255 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 06 04:21:01.986255 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 04:21:01.986255 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 618/1253] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Sep 6 04:21:01 2022] GET /identity/v3/users/placement => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 04:21:01.991619 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997db285-ab25-4e64-9c53-ab61fec08ffa None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:21:02.048602 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:21:02.092764 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997db285-ab25-4e64-9c53-ab61fec08ffa None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:21:02.132790 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9be990-cd2b-4e5a-a49e-d0d1b5521e53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:21:02.133489 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9be990-cd2b-4e5a-a49e-d0d1b5521e53 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:21:02.133769 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9be990-cd2b-4e5a-a49e-d0d1b5521e53 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:21:02.134040 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9be990-cd2b-4e5a-a49e-d0d1b5521e53 None admin] PATH_INFO: `/v3/users` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:21:02.138009 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c9be990-cd2b-4e5a-a49e-d0d1b5521e53 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:21:02.138009 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c9be990-cd2b-4e5a-a49e-d0d1b5521e53 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:21:02.138009 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c9be990-cd2b-4e5a-a49e-d0d1b5521e53 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:21:02.154870 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 636/1254] 10.222.0.30 () {56 vars in 1143 bytes} [Tue Sep 6 04:21:01 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:21:02.160964 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6713c2-3c04-48c6-a6e0-481e708c04ed None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:21:02.262022 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6713c2-3c04-48c6-a6e0-481e708c04ed None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:21:02.295095 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4f7b28-1377-40f5-9d4e-5bc6f62992a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:21:02.295703 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4f7b28-1377-40f5-9d4e-5bc6f62992a5 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:21:02.295983 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4f7b28-1377-40f5-9d4e-5bc6f62992a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:21:02.296298 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4f7b28-1377-40f5-9d4e-5bc6f62992a5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:21:02.352122 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:21:02.364536 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4f7b28-1377-40f5-9d4e-5bc6f62992a5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:21:02.371696 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4f7b28-1377-40f5-9d4e-5bc6f62992a5 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:21:02.463163 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 619/1255] 10.222.0.30 () {58 vars in 1282 bytes} [Tue Sep 6 04:21:02 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 06 04:21:02.476114 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9be990-cd2b-4e5a-a49e-d0d1b5521e53 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:21:02.581645 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9be990-cd2b-4e5a-a49e-d0d1b5521e53 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:21:02.595360 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:21:02.616164 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c1390f-6bd3-426c-89e2-1df390fe5e25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:21:02.616732 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c1390f-6bd3-426c-89e2-1df390fe5e25 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:21:02.617006 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c1390f-6bd3-426c-89e2-1df390fe5e25 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:21:02.617273 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c1390f-6bd3-426c-89e2-1df390fe5e25 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:21:02.627218 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c1390f-6bd3-426c-89e2-1df390fe5e25 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:21:02.628655 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c1390f-6bd3-426c-89e2-1df390fe5e25 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:21:02.643410 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-41c1390f-6bd3-426c-89e2-1df390fe5e25 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 04:21:02.643410 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 04:21:02.643410 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 04:21:02.643410 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 04:21:02.643410 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 04:21:02.643410 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 04:21:02.643410 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 04:21:02.643410 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 04:21:02.643410 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 04:21:02.643410 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 04:21:02.643410 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 04:21:02.643410 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 04:21:02.643410 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 04:21:02.643410 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 04:21:02.643410 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 04:21:02.643410 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 04:21:02.644541 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 04:21:02.644541 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 04:21:02.644541 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 04:21:02.644541 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 04:21:02.644541 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 04:21:02.644541 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 04:21:02.644541 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 04:21:02.644541 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 04:21:02.644541 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 04:21:02.644541 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 04:21:02.644541 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 04:21:02.644541 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 04:21:02.644541 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 04:21:02.644541 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 04:21:02.644541 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 04:21:02.644541 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 04:21:02.644541 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 04:21:02.645771 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 04:21:02.645771 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 04:21:02.645771 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 04:21:02.649183 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 637/1256] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Sep 6 04:21:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 04:21:02.650251 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4f7b28-1377-40f5-9d4e-5bc6f62992a5 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:21:02.737384 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:21:02.748874 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4f7b28-1377-40f5-9d4e-5bc6f62992a5 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:21:02.785227 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c214f636-7be8-44a9-86c6-29a3ea904ca5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:21:02.785897 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c214f636-7be8-44a9-86c6-29a3ea904ca5 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:21:02.786206 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c214f636-7be8-44a9-86c6-29a3ea904ca5 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:21:02.786472 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c214f636-7be8-44a9-86c6-29a3ea904ca5 None admin] PATH_INFO: `/v3/domains` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:21:02.787106 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c214f636-7be8-44a9-86c6-29a3ea904ca5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:21:02.787401 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c214f636-7be8-44a9-86c6-29a3ea904ca5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:21:02.788747 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c214f636-7be8-44a9-86c6-29a3ea904ca5 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:21:02.799791 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 620/1257] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Sep 6 04:21:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:21:02.805473 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c1390f-6bd3-426c-89e2-1df390fe5e25 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:21:02.905615 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c1390f-6bd3-426c-89e2-1df390fe5e25 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:21:02.938878 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9809fc90-af9e-4eed-ad06-9fee58832933 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:21:02.939508 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9809fc90-af9e-4eed-ad06-9fee58832933 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:21:02.939791 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9809fc90-af9e-4eed-ad06-9fee58832933 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:21:02.940063 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9809fc90-af9e-4eed-ad06-9fee58832933 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:21:02.946583 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9809fc90-af9e-4eed-ad06-9fee58832933 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:21:02.948028 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9809fc90-af9e-4eed-ad06-9fee58832933 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:21:02.959843 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-9809fc90-af9e-4eed-ad06-9fee58832933 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 04:21:02.959843 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 04:21:02.959843 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 04:21:02.959843 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 04:21:02.959843 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 04:21:02.959843 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 04:21:02.959843 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 04:21:02.959843 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 04:21:02.959843 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 04:21:02.959843 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 04:21:02.959843 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 04:21:02.959843 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 04:21:02.959843 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 06 04:21:02.959843 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 06 04:21:02.959843 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 06 04:21:02.960911 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 06 04:21:02.960911 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 04:21:02.960911 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 04:21:02.960911 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 04:21:02.960911 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 04:21:02.960911 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 04:21:02.960911 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 04:21:02.960911 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 04:21:02.960911 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 04:21:02.960911 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 04:21:02.960911 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 04:21:02.960911 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 04:21:02.960911 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 04:21:02.960911 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 04:21:02.960911 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 04:21:02.960911 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 04:21:02.960911 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 04:21:02.962123 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 06 04:21:02.962123 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 06 04:21:02.962123 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 06 04:21:02.962123 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 06 04:21:02.962123 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 06 04:21:02.962123 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 06 04:21:02.962123 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 04:21:02.962123 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 04:21:02.962123 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 638/1258] 10.222.0.30 () {56 vars in 1111 bytes} [Tue Sep 6 04:21:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 04:21:02.968664 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c214f636-7be8-44a9-86c6-29a3ea904ca5 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:21:03.068402 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c214f636-7be8-44a9-86c6-29a3ea904ca5 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:21:03.102276 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5e04ea-5bfa-4bcb-910f-491d27d6118c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:21:03.102897 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5e04ea-5bfa-4bcb-910f-491d27d6118c None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:21:03.103180 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5e04ea-5bfa-4bcb-910f-491d27d6118c None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:21:03.103455 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5e04ea-5bfa-4bcb-910f-491d27d6118c None admin] PATH_INFO: `/v3/projects` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:21:03.104147 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5e04ea-5bfa-4bcb-910f-491d27d6118c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:21:03.104451 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5e04ea-5bfa-4bcb-910f-491d27d6118c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:21:03.105974 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5e04ea-5bfa-4bcb-910f-491d27d6118c None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:21:03.119211 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 621/1259] 10.222.0.30 () {56 vars in 1148 bytes} [Tue Sep 6 04:21:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:21:03.125333 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9809fc90-af9e-4eed-ad06-9fee58832933 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:21:03.152168 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:21:03.225243 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9809fc90-af9e-4eed-ad06-9fee58832933 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:21:03.259321 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae37fe4f-6910-41f0-aabb-164b4a1ea946 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:21:03.259920 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae37fe4f-6910-41f0-aabb-164b4a1ea946 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:21:03.260201 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae37fe4f-6910-41f0-aabb-164b4a1ea946 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:21:03.260475 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae37fe4f-6910-41f0-aabb-164b4a1ea946 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:21:03.261200 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae37fe4f-6910-41f0-aabb-164b4a1ea946 None admin] RBAC: Adding query filter params (role.id=ea4c814713b149108e488de855b025e4, scope.project.id=b707733ce65c42edbc1fd95ad0a1c06c, user.id=1d4f2e4dbd41441d84f03ddb30191e7a) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:21:03.261535 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae37fe4f-6910-41f0-aabb-164b4a1ea946 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:21:03.262975 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae37fe4f-6910-41f0-aabb-164b4a1ea946 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:21:03.268345 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 639/1260] 10.222.0.30 () {56 vars in 1374 bytes} [Tue Sep 6 04:21:03 2022] GET /identity/v3/role_assignments?user.id=1d4f2e4dbd41441d84f03ddb30191e7a&scope.project.id=b707733ce65c42edbc1fd95ad0a1c06c&role.id=ea4c814713b149108e488de855b025e4 => generated 259 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:21:05.160788 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441cee55-4769-49e5-8aaf-2a5dcd947e68 None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:21:05.161382 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441cee55-4769-49e5-8aaf-2a5dcd947e68 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:21:05.161495 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441cee55-4769-49e5-8aaf-2a5dcd947e68 None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:21:05.163000 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 622/1261] 10.222.0.30 () {54 vars in 927 bytes} [Tue Sep 6 04:21:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 04:21:05.168341 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3c7f27-45c0-442f-b763-61c72f3dcb52 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:21:05.168761 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3c7f27-45c0-442f-b763-61c72f3dcb52 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:21:05.168887 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3c7f27-45c0-442f-b763-61c72f3dcb52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:21:05.187941 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-4a3c7f27-45c0-442f-b763-61c72f3dcb52 None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:21:05.189722 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-4a3c7f27-45c0-442f-b763-61c72f3dcb52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:21:05.256250 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 640/1262] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Sep 6 04:21:05 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 04:21:05.533711 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441cee55-4769-49e5-8aaf-2a5dcd947e68 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:21:05.544857 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:21:05.637165 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441cee55-4769-49e5-8aaf-2a5dcd947e68 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:21:05.671161 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bee67b8-0ab0-4624-b9d4-92f8f4068c3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:21:05.671794 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bee67b8-0ab0-4624-b9d4-92f8f4068c3b None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:21:05.672074 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bee67b8-0ab0-4624-b9d4-92f8f4068c3b None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:21:05.672896 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bee67b8-0ab0-4624-b9d4-92f8f4068c3b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:21:05.683936 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bee67b8-0ab0-4624-b9d4-92f8f4068c3b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 04:21:05.689777 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-6bee67b8-0ab0-4624-b9d4-92f8f4068c3b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 04:21:05.689777 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 04:21:05.689777 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 04:21:05.689777 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 04:21:05.689777 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 04:21:05.689777 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 04:21:05.689777 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 04:21:05.689777 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 04:21:05.689777 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 04:21:05.689777 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 04:21:05.689777 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 04:21:05.689777 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 04:21:05.689777 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 04:21:05.689777 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 04:21:05.689777 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 04:21:05.689777 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 04:21:05.690942 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 04:21:05.690942 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 04:21:05.690942 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 04:21:05.690942 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 04:21:05.690942 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 04:21:05.690942 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 04:21:05.690942 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 04:21:05.690942 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 04:21:05.690942 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 04:21:05.690942 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 04:21:05.690942 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 04:21:05.690942 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 04:21:05.690942 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 04:21:05.690942 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 04:21:05.690942 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 04:21:05.690942 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 04:21:05.691934 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 04:21:05.691934 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 04:21:05.691934 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 04:21:05.691934 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 04:21:05.691934 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 04:21:05.691934 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 04:21:05.691934 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 04:21:05.691934 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 04:21:05.691934 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 04:21:05.691934 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 04:21:05.691934 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 04:21:05.691934 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 04:21:05.691934 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 623/1263] 10.222.0.30 () {56 vars in 1096 bytes} [Tue Sep 6 04:21:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 04:21:05.696906 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3c7f27-45c0-442f-b763-61c72f3dcb52 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:21:05.714766 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:21:05.793225 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3c7f27-45c0-442f-b763-61c72f3dcb52 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:21:05.829251 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195c0639-174c-4f35-9453-85751e982548 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:21:05.829992 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195c0639-174c-4f35-9453-85751e982548 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:21:05.830271 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195c0639-174c-4f35-9453-85751e982548 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:21:05.830540 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195c0639-174c-4f35-9453-85751e982548 None admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:21:05.831209 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195c0639-174c-4f35-9453-85751e982548 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:21:05.831496 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195c0639-174c-4f35-9453-85751e982548 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:21:05.832871 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195c0639-174c-4f35-9453-85751e982548 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:21:05.840818 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 641/1264] 10.222.0.30 () {56 vars in 1099 bytes} [Tue Sep 6 04:21:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:21:05.847178 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bee67b8-0ab0-4624-b9d4-92f8f4068c3b None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:21:05.919246 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:21:05.951262 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bee67b8-0ab0-4624-b9d4-92f8f4068c3b None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:21:05.989330 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71b984e-3570-4544-8c44-dc8888fc4607 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:21:05.989330 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71b984e-3570-4544-8c44-dc8888fc4607 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:21:05.989330 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71b984e-3570-4544-8c44-dc8888fc4607 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:21:05.989330 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71b984e-3570-4544-8c44-dc8888fc4607 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:21:05.996974 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71b984e-3570-4544-8c44-dc8888fc4607 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:21:05.998438 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71b984e-3570-4544-8c44-dc8888fc4607 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:21:06.012925 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-b71b984e-3570-4544-8c44-dc8888fc4607 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 04:21:06.012925 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 04:21:06.012925 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 04:21:06.012925 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 04:21:06.012925 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 04:21:06.012925 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 04:21:06.012925 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 04:21:06.012925 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 04:21:06.012925 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 04:21:06.012925 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 04:21:06.012925 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 04:21:06.012925 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 04:21:06.012925 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 04:21:06.012925 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 04:21:06.012925 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 04:21:06.012925 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 04:21:06.014210 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 04:21:06.014210 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 04:21:06.014210 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 04:21:06.014210 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 04:21:06.014210 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 04:21:06.014210 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 04:21:06.014210 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 04:21:06.014210 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 04:21:06.014210 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 04:21:06.014210 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 04:21:06.014210 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 04:21:06.014210 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 04:21:06.014210 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 04:21:06.014210 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 04:21:06.014210 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 04:21:06.014210 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 04:21:06.014210 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 04:21:06.015157 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 04:21:06.015157 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 04:21:06.015157 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 04:21:06.017463 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 624/1265] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Sep 6 04:21:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 04:21:06.020003 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195c0639-174c-4f35-9453-85751e982548 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:21:06.101857 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:21:06.120248 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195c0639-174c-4f35-9453-85751e982548 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:21:06.153940 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d171792-789f-4a6a-938a-c8b53de8668b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:21:06.154545 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d171792-789f-4a6a-938a-c8b53de8668b None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:21:06.154828 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d171792-789f-4a6a-938a-c8b53de8668b None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:21:06.155103 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d171792-789f-4a6a-938a-c8b53de8668b None admin] PATH_INFO: `/v3/domains` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:21:06.155743 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d171792-789f-4a6a-938a-c8b53de8668b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:21:06.156046 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d171792-789f-4a6a-938a-c8b53de8668b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:21:06.158133 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d171792-789f-4a6a-938a-c8b53de8668b None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:21:06.168937 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 642/1266] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Sep 6 04:21:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:21:06.175347 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71b984e-3570-4544-8c44-dc8888fc4607 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:21:06.277972 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71b984e-3570-4544-8c44-dc8888fc4607 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:21:06.297198 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:21:06.311122 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01840f26-a374-44eb-b184-d4fdff847ef2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:21:06.311907 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01840f26-a374-44eb-b184-d4fdff847ef2 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:21:06.312179 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01840f26-a374-44eb-b184-d4fdff847ef2 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:21:06.312447 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01840f26-a374-44eb-b184-d4fdff847ef2 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:21:06.320379 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01840f26-a374-44eb-b184-d4fdff847ef2 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:21:06.321875 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01840f26-a374-44eb-b184-d4fdff847ef2 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:21:06.332635 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-01840f26-a374-44eb-b184-d4fdff847ef2 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 06 04:21:06.332635 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 04:21:06.332635 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 04:21:06.332635 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 04:21:06.332635 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 04:21:06.332635 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 04:21:06.332635 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 04:21:06.332635 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 04:21:06.332635 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 04:21:06.332635 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 04:21:06.332635 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 04:21:06.332635 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 04:21:06.332635 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 06 04:21:06.332635 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 06 04:21:06.332635 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 06 04:21:06.332635 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 06 04:21:06.333767 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 04:21:06.333767 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 04:21:06.333767 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 06 04:21:06.333767 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 04:21:06.333767 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 06 04:21:06.333767 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 04:21:06.333767 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 04:21:06.333767 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 04:21:06.333767 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 04:21:06.333767 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 04:21:06.333767 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 04:21:06.333767 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 04:21:06.333767 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 04:21:06.333767 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 04:21:06.333767 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 04:21:06.333767 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 04:21:06.333767 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 04:21:06.333767 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 04:21:06.334759 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 04:21:06.334759 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 04:21:06.334759 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 06 04:21:06.334759 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 06 04:21:06.334759 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 06 04:21:06.334759 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 06 04:21:06.334759 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 06 04:21:06.334759 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 06 04:21:06.334759 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 06 04:21:06.334759 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 04:21:06.334759 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 625/1267] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Sep 6 04:21:06 2022] GET /identity/v3/users/placement => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 04:21:06.339891 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d171792-789f-4a6a-938a-c8b53de8668b None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:21:06.441401 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d171792-789f-4a6a-938a-c8b53de8668b None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:21:06.475842 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32dd3e7-5f9c-4523-ba20-4c39058293a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:21:06.476451 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32dd3e7-5f9c-4523-ba20-4c39058293a8 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:21:06.476729 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32dd3e7-5f9c-4523-ba20-4c39058293a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:21:06.476997 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32dd3e7-5f9c-4523-ba20-4c39058293a8 None admin] PATH_INFO: `/v3/users` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:21:06.477774 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32dd3e7-5f9c-4523-ba20-4c39058293a8 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:21:06.478079 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32dd3e7-5f9c-4523-ba20-4c39058293a8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:21:06.479513 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32dd3e7-5f9c-4523-ba20-4c39058293a8 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:21:06.489429 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:21:06.497190 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 643/1268] 10.222.0.30 () {56 vars in 1143 bytes} [Tue Sep 6 04:21:06 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:21:06.504145 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01840f26-a374-44eb-b184-d4fdff847ef2 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:21:06.612284 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01840f26-a374-44eb-b184-d4fdff847ef2 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:21:06.646910 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f10513-1db3-465e-94ee-bf1904784669 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:21:06.647481 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f10513-1db3-465e-94ee-bf1904784669 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:21:06.647763 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f10513-1db3-465e-94ee-bf1904784669 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:21:06.648037 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f10513-1db3-465e-94ee-bf1904784669 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:21:06.658816 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f10513-1db3-465e-94ee-bf1904784669 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:21:06.660258 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f10513-1db3-465e-94ee-bf1904784669 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:21:06.674469 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-71f10513-1db3-465e-94ee-bf1904784669 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 04:21:06.674469 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 04:21:06.674469 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 04:21:06.674469 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 04:21:06.674469 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 04:21:06.674469 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 04:21:06.674469 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 04:21:06.674469 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 04:21:06.674469 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 04:21:06.674469 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 04:21:06.674469 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 04:21:06.674469 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 04:21:06.674469 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 04:21:06.674469 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 04:21:06.674469 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 04:21:06.674469 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 04:21:06.676883 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 04:21:06.676883 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 04:21:06.676883 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 04:21:06.676883 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 04:21:06.676883 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 04:21:06.676883 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 04:21:06.676883 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 04:21:06.676883 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 04:21:06.676883 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 04:21:06.676883 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 04:21:06.676883 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 04:21:06.676883 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 04:21:06.676883 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 04:21:06.676883 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 04:21:06.676883 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 04:21:06.676883 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 04:21:06.676883 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 04:21:06.678018 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 04:21:06.678018 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 04:21:06.678018 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 04:21:06.678018 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 626/1269] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Sep 6 04:21:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 04:21:06.681735 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32dd3e7-5f9c-4523-ba20-4c39058293a8 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:21:06.783418 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32dd3e7-5f9c-4523-ba20-4c39058293a8 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:21:06.818361 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658be083-6c38-476f-a030-60d53d755361 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:21:06.818966 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658be083-6c38-476f-a030-60d53d755361 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:21:06.819245 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658be083-6c38-476f-a030-60d53d755361 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:21:06.819511 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658be083-6c38-476f-a030-60d53d755361 None admin] PATH_INFO: `/v3/domains` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:21:06.820178 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658be083-6c38-476f-a030-60d53d755361 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:21:06.820479 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658be083-6c38-476f-a030-60d53d755361 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:21:06.821893 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658be083-6c38-476f-a030-60d53d755361 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:21:06.833042 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 644/1270] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Sep 6 04:21:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:21:06.839116 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f10513-1db3-465e-94ee-bf1904784669 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:21:06.872015 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:21:06.941242 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f10513-1db3-465e-94ee-bf1904784669 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:21:06.974890 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f3d1bb-a1d3-4f2e-8869-9c72af018e13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:21:06.975521 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f3d1bb-a1d3-4f2e-8869-9c72af018e13 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:21:06.975802 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f3d1bb-a1d3-4f2e-8869-9c72af018e13 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:21:06.976107 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f3d1bb-a1d3-4f2e-8869-9c72af018e13 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:21:06.982575 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f3d1bb-a1d3-4f2e-8869-9c72af018e13 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:21:06.984007 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f3d1bb-a1d3-4f2e-8869-9c72af018e13 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:21:06.995524 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-37f3d1bb-a1d3-4f2e-8869-9c72af018e13 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 04:21:06.995524 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 04:21:06.995524 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 04:21:06.995524 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 04:21:06.995524 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 04:21:06.995524 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 04:21:06.995524 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 04:21:06.995524 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 04:21:06.995524 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 04:21:06.995524 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 04:21:06.995524 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 04:21:06.995524 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 04:21:06.995524 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 06 04:21:06.995524 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 06 04:21:06.995524 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 06 04:21:06.998617 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 06 04:21:06.998617 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 04:21:06.998617 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 04:21:06.998617 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 04:21:06.998617 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 04:21:06.998617 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 04:21:06.998617 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 04:21:06.998617 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 04:21:06.998617 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 04:21:06.998617 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 04:21:06.998617 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 04:21:06.998617 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 04:21:06.998617 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 04:21:06.998617 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 04:21:06.998617 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 04:21:06.998617 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 04:21:06.998617 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 04:21:06.999777 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 06 04:21:06.999777 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 06 04:21:06.999777 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 06 04:21:06.999777 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 06 04:21:06.999777 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 06 04:21:06.999777 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 06 04:21:06.999777 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 04:21:06.999777 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 04:21:06.999777 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 627/1271] 10.222.0.30 () {56 vars in 1111 bytes} [Tue Sep 6 04:21:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 04:21:07.002913 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658be083-6c38-476f-a030-60d53d755361 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:21:07.062452 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:21:07.103003 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658be083-6c38-476f-a030-60d53d755361 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:21:07.136659 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23417d9-f523-41a4-a096-1b803ab425fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:21:07.137270 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23417d9-f523-41a4-a096-1b803ab425fd None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:21:07.137574 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23417d9-f523-41a4-a096-1b803ab425fd None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:21:07.137844 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23417d9-f523-41a4-a096-1b803ab425fd None admin] PATH_INFO: `/v3/projects` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:21:07.138529 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23417d9-f523-41a4-a096-1b803ab425fd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:21:07.138830 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23417d9-f523-41a4-a096-1b803ab425fd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:21:07.140295 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23417d9-f523-41a4-a096-1b803ab425fd None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:21:07.152275 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 645/1272] 10.222.0.30 () {56 vars in 1148 bytes} [Tue Sep 6 04:21:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:21:07.158486 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f3d1bb-a1d3-4f2e-8869-9c72af018e13 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:21:07.251187 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:21:07.259738 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f3d1bb-a1d3-4f2e-8869-9c72af018e13 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:21:07.294798 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a028f69-ed05-47d5-8a6d-6e7c085917c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:21:07.295619 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a028f69-ed05-47d5-8a6d-6e7c085917c0 None admin] REQUEST_METHOD: `PUT` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:21:07.296277 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a028f69-ed05-47d5-8a6d-6e7c085917c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:21:07.296552 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a028f69-ed05-47d5-8a6d-6e7c085917c0 None admin] PATH_INFO: `/v3/projects/b707733ce65c42edbc1fd95ad0a1c06c/users/1d4f2e4dbd41441d84f03ddb30191e7a/roles/ea4c814713b149108e488de855b025e4` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:21:07.299055 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a028f69-ed05-47d5-8a6d-6e7c085917c0 None admin] RBAC: Authorizing `identity:create_grant(project_id=b707733ce65c42edbc1fd95ad0a1c06c, user_id=1d4f2e4dbd41441d84f03ddb30191e7a, role_id=ea4c814713b149108e488de855b025e4)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:21:07.300463 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a028f69-ed05-47d5-8a6d-6e7c085917c0 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:21:07.427463 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 628/1273] 10.222.0.30 () {58 vars in 1439 bytes} [Tue Sep 6 04:21:07 2022] PUT /identity/v3/projects/b707733ce65c42edbc1fd95ad0a1c06c/users/1d4f2e4dbd41441d84f03ddb30191e7a/roles/ea4c814713b149108e488de855b025e4 => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 04:21:09.334188 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4ed100-0fd8-4577-9220-2069e5ff3416 None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:21:09.334745 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4ed100-0fd8-4577-9220-2069e5ff3416 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:21:09.334745 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4ed100-0fd8-4577-9220-2069e5ff3416 None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:21:09.336113 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 646/1274] 10.222.0.30 () {54 vars in 927 bytes} [Tue Sep 6 04:21:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 04:21:09.341107 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e876d22-6453-4d01-959e-0e72188d2a06 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:21:09.341622 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e876d22-6453-4d01-959e-0e72188d2a06 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:21:09.341709 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e876d22-6453-4d01-959e-0e72188d2a06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:21:09.360803 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-4e876d22-6453-4d01-959e-0e72188d2a06 None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:21:09.362119 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-4e876d22-6453-4d01-959e-0e72188d2a06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:21:09.429606 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 629/1275] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Sep 6 04:21:09 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 04:21:09.700485 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4ed100-0fd8-4577-9220-2069e5ff3416 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:21:09.801635 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4ed100-0fd8-4577-9220-2069e5ff3416 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:21:09.822586 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:21:09.841597 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5a44a1-8fb1-44ac-8944-7173e22d6143 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:21:09.842229 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5a44a1-8fb1-44ac-8944-7173e22d6143 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:21:09.842503 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5a44a1-8fb1-44ac-8944-7173e22d6143 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:21:09.842768 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5a44a1-8fb1-44ac-8944-7173e22d6143 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:21:09.853151 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb5a44a1-8fb1-44ac-8944-7173e22d6143 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 04:21:09.857947 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-bb5a44a1-8fb1-44ac-8944-7173e22d6143 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 04:21:09.857947 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 04:21:09.857947 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 04:21:09.857947 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 04:21:09.857947 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 04:21:09.857947 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 04:21:09.857947 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 04:21:09.857947 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 04:21:09.857947 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 04:21:09.857947 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 04:21:09.857947 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 04:21:09.857947 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 04:21:09.857947 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 04:21:09.857947 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 04:21:09.857947 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 04:21:09.857947 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 04:21:09.862112 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 04:21:09.862112 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 04:21:09.862112 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 04:21:09.862112 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 04:21:09.862112 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 04:21:09.862112 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 04:21:09.862112 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 04:21:09.862112 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 04:21:09.862112 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 04:21:09.862112 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 04:21:09.862112 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 04:21:09.862112 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 04:21:09.862112 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 04:21:09.862112 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 04:21:09.862112 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 04:21:09.862112 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 04:21:09.863194 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 04:21:09.863194 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 04:21:09.863194 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 04:21:09.863194 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 04:21:09.863194 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 04:21:09.863194 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 04:21:09.863194 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 04:21:09.863194 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 04:21:09.863194 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 04:21:09.863194 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 04:21:09.863194 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 04:21:09.863194 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 04:21:09.863194 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 647/1276] 10.222.0.30 () {56 vars in 1096 bytes} [Tue Sep 6 04:21:09 2022] GET /identity/v3/roles/admin => generated 83 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 04:21:09.866034 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e876d22-6453-4d01-959e-0e72188d2a06 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:21:09.946331 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:21:09.964505 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e876d22-6453-4d01-959e-0e72188d2a06 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:21:09.999491 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2333b3a5-97d4-407e-b1f5-b64df4c0754a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:21:10.000098 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2333b3a5-97d4-407e-b1f5-b64df4c0754a None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:21:10.000399 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2333b3a5-97d4-407e-b1f5-b64df4c0754a None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:21:10.000894 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2333b3a5-97d4-407e-b1f5-b64df4c0754a None admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:21:10.001967 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2333b3a5-97d4-407e-b1f5-b64df4c0754a None admin] RBAC: Adding query filter params (name=admin) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:21:10.002301 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2333b3a5-97d4-407e-b1f5-b64df4c0754a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:21:10.003664 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2333b3a5-97d4-407e-b1f5-b64df4c0754a None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:21:10.012028 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 630/1277] 10.222.0.30 () {56 vars in 1099 bytes} [Tue Sep 6 04:21:09 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:21:10.018783 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5a44a1-8fb1-44ac-8944-7173e22d6143 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:21:10.121145 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5a44a1-8fb1-44ac-8944-7173e22d6143 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:21:10.155785 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac71234f-092b-4724-9c20-268bb50e4814 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:21:10.156417 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac71234f-092b-4724-9c20-268bb50e4814 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:21:10.157339 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac71234f-092b-4724-9c20-268bb50e4814 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:21:10.157619 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac71234f-092b-4724-9c20-268bb50e4814 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:21:10.167605 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac71234f-092b-4724-9c20-268bb50e4814 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:21:10.169090 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac71234f-092b-4724-9c20-268bb50e4814 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:21:10.183549 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-ac71234f-092b-4724-9c20-268bb50e4814 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 04:21:10.183549 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 04:21:10.183549 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 04:21:10.183549 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 04:21:10.183549 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 04:21:10.183549 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 04:21:10.183549 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 04:21:10.183549 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 04:21:10.183549 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 04:21:10.183549 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 04:21:10.183549 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 04:21:10.183549 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 04:21:10.183549 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 04:21:10.183549 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 04:21:10.183549 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 04:21:10.183549 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 04:21:10.186451 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 04:21:10.186451 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 04:21:10.186451 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 04:21:10.186451 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 04:21:10.186451 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 04:21:10.186451 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 04:21:10.186451 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 04:21:10.186451 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 04:21:10.186451 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 04:21:10.186451 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 04:21:10.186451 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 04:21:10.186451 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 04:21:10.186451 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 04:21:10.186451 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 04:21:10.186451 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 04:21:10.186451 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 04:21:10.186451 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 04:21:10.187600 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 04:21:10.187600 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 04:21:10.187600 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 04:21:10.187600 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 648/1278] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Sep 6 04:21:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 04:21:10.191040 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2333b3a5-97d4-407e-b1f5-b64df4c0754a None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:21:10.290507 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2333b3a5-97d4-407e-b1f5-b64df4c0754a None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:21:10.323330 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d39941-4214-4e7f-8d36-2e09ca121fed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:21:10.323938 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d39941-4214-4e7f-8d36-2e09ca121fed None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:21:10.324218 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d39941-4214-4e7f-8d36-2e09ca121fed None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:21:10.324487 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d39941-4214-4e7f-8d36-2e09ca121fed None admin] PATH_INFO: `/v3/domains` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:21:10.325119 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d39941-4214-4e7f-8d36-2e09ca121fed None admin] RBAC: Adding query filter params (name=Default) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:21:10.325449 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d39941-4214-4e7f-8d36-2e09ca121fed None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:21:10.326805 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d39941-4214-4e7f-8d36-2e09ca121fed None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:21:10.338954 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 631/1279] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Sep 6 04:21:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:21:10.345367 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac71234f-092b-4724-9c20-268bb50e4814 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:21:10.377453 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:21:10.445383 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac71234f-092b-4724-9c20-268bb50e4814 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:21:10.479362 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f892e29-0dad-4268-a9f1-ff17c3de2d6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:21:10.479991 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f892e29-0dad-4268-a9f1-ff17c3de2d6e None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:21:10.480270 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f892e29-0dad-4268-a9f1-ff17c3de2d6e None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:21:10.480543 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f892e29-0dad-4268-a9f1-ff17c3de2d6e None admin] PATH_INFO: `/v3/users/placement` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:21:10.488182 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f892e29-0dad-4268-a9f1-ff17c3de2d6e None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:21:10.489675 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f892e29-0dad-4268-a9f1-ff17c3de2d6e None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:21:10.503581 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-3f892e29-0dad-4268-a9f1-ff17c3de2d6e None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 06 04:21:10.503581 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 04:21:10.503581 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 04:21:10.503581 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 04:21:10.503581 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 04:21:10.503581 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 04:21:10.503581 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 04:21:10.503581 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 04:21:10.503581 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 04:21:10.503581 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 04:21:10.503581 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 04:21:10.503581 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 04:21:10.503581 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 06 04:21:10.503581 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 06 04:21:10.503581 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 06 04:21:10.503581 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 06 04:21:10.509439 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 04:21:10.509439 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 04:21:10.509439 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 06 04:21:10.509439 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 04:21:10.509439 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 06 04:21:10.509439 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 04:21:10.509439 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 04:21:10.509439 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 04:21:10.509439 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 04:21:10.509439 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 04:21:10.509439 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 04:21:10.509439 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 04:21:10.509439 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 04:21:10.509439 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 04:21:10.509439 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 04:21:10.509439 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 04:21:10.509439 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 04:21:10.509439 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 04:21:10.510623 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 04:21:10.510623 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 04:21:10.510623 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 06 04:21:10.510623 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 06 04:21:10.510623 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 06 04:21:10.510623 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 06 04:21:10.510623 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 06 04:21:10.510623 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 06 04:21:10.510623 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 06 04:21:10.510623 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 04:21:10.510623 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 649/1280] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Sep 6 04:21:10 2022] GET /identity/v3/users/placement => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 04:21:10.511384 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d39941-4214-4e7f-8d36-2e09ca121fed None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:21:10.514920 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:21:10.619003 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d39941-4214-4e7f-8d36-2e09ca121fed None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:21:10.656267 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8276d9a4-f8d3-47f4-a417-1488dd9cd4fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:21:10.656991 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8276d9a4-f8d3-47f4-a417-1488dd9cd4fd None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:21:10.657273 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8276d9a4-f8d3-47f4-a417-1488dd9cd4fd None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:21:10.657576 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8276d9a4-f8d3-47f4-a417-1488dd9cd4fd None admin] PATH_INFO: `/v3/users` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:21:10.658335 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8276d9a4-f8d3-47f4-a417-1488dd9cd4fd None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:21:10.658636 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8276d9a4-f8d3-47f4-a417-1488dd9cd4fd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:21:10.660115 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8276d9a4-f8d3-47f4-a417-1488dd9cd4fd None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:21:10.677481 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 632/1281] 10.222.0.30 () {56 vars in 1143 bytes} [Tue Sep 6 04:21:10 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:21:10.683360 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f892e29-0dad-4268-a9f1-ff17c3de2d6e None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:21:10.775391 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:21:10.785679 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f892e29-0dad-4268-a9f1-ff17c3de2d6e None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:21:10.818327 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce55cee-576e-483c-a039-9cd15345a574 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:21:10.818926 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce55cee-576e-483c-a039-9cd15345a574 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:21:10.819280 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce55cee-576e-483c-a039-9cd15345a574 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:21:10.819560 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce55cee-576e-483c-a039-9cd15345a574 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:21:10.892029 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce55cee-576e-483c-a039-9cd15345a574 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:21:10.893477 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce55cee-576e-483c-a039-9cd15345a574 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:21:10.995255 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:21:11.007548 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 650/1282] 10.222.0.30 () {58 vars in 1282 bytes} [Tue Sep 6 04:21:10 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 06 04:21:11.013676 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8276d9a4-f8d3-47f4-a417-1488dd9cd4fd None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:21:11.080187 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:21:11.131598 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8276d9a4-f8d3-47f4-a417-1488dd9cd4fd None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:21:11.169203 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5e7a83-287e-4120-b4ae-52ff4b4911eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:21:11.169802 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5e7a83-287e-4120-b4ae-52ff4b4911eb None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:21:11.170085 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5e7a83-287e-4120-b4ae-52ff4b4911eb None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:21:11.170357 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5e7a83-287e-4120-b4ae-52ff4b4911eb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:21:11.181421 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca5e7a83-287e-4120-b4ae-52ff4b4911eb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:21:11.182895 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca5e7a83-287e-4120-b4ae-52ff4b4911eb None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:21:11.199076 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-ca5e7a83-287e-4120-b4ae-52ff4b4911eb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 04:21:11.199076 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 04:21:11.199076 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 04:21:11.199076 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 04:21:11.199076 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 04:21:11.199076 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 04:21:11.199076 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 04:21:11.199076 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 04:21:11.199076 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 04:21:11.199076 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 04:21:11.199076 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 04:21:11.199076 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 04:21:11.199076 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 04:21:11.199076 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 04:21:11.199076 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 04:21:11.199076 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 04:21:11.200306 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 04:21:11.200306 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 04:21:11.200306 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 04:21:11.200306 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 04:21:11.200306 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 04:21:11.200306 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 04:21:11.200306 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 04:21:11.200306 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 04:21:11.200306 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 04:21:11.200306 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 04:21:11.200306 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 04:21:11.200306 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 04:21:11.200306 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 04:21:11.200306 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 04:21:11.200306 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 04:21:11.200306 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 04:21:11.200306 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 04:21:11.206018 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 04:21:11.206018 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 04:21:11.206018 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 04:21:11.206018 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 633/1283] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Sep 6 04:21:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 04:21:11.208513 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce55cee-576e-483c-a039-9cd15345a574 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:21:11.312322 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce55cee-576e-483c-a039-9cd15345a574 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:21:11.347738 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b2cf28-a985-4d68-8dec-26654eda8265 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:21:11.348408 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b2cf28-a985-4d68-8dec-26654eda8265 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:21:11.348642 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b2cf28-a985-4d68-8dec-26654eda8265 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:21:11.348912 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b2cf28-a985-4d68-8dec-26654eda8265 None admin] PATH_INFO: `/v3/domains` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:21:11.349585 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b2cf28-a985-4d68-8dec-26654eda8265 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:21:11.349900 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b2cf28-a985-4d68-8dec-26654eda8265 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:21:11.351252 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b2cf28-a985-4d68-8dec-26654eda8265 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:21:11.363295 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 651/1284] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Sep 6 04:21:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:21:11.369036 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5e7a83-287e-4120-b4ae-52ff4b4911eb None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:21:11.473073 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5e7a83-287e-4120-b4ae-52ff4b4911eb None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:21:11.491886 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:21:11.517920 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308eb35a-82e2-4f85-b392-9dab1b7ebd56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:21:11.518954 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308eb35a-82e2-4f85-b392-9dab1b7ebd56 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:21:11.518954 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308eb35a-82e2-4f85-b392-9dab1b7ebd56 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:21:11.519285 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308eb35a-82e2-4f85-b392-9dab1b7ebd56 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:21:11.527798 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308eb35a-82e2-4f85-b392-9dab1b7ebd56 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:21:11.529278 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308eb35a-82e2-4f85-b392-9dab1b7ebd56 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:21:11.541533 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application [None req-308eb35a-82e2-4f85-b392-9dab1b7ebd56 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 04:21:11.541533 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 04:21:11.541533 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 04:21:11.541533 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 04:21:11.541533 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 04:21:11.541533 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 04:21:11.541533 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 04:21:11.541533 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 04:21:11.541533 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 04:21:11.541533 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 04:21:11.541533 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 04:21:11.541533 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 04:21:11.541533 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 06 04:21:11.541533 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 06 04:21:11.541533 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 06 04:21:11.542617 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 06 04:21:11.542617 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 04:21:11.542617 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 04:21:11.542617 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 04:21:11.542617 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 04:21:11.542617 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 04:21:11.542617 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 04:21:11.542617 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 04:21:11.542617 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application with Lock( Sep 06 04:21:11.542617 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 04:21:11.542617 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._enter() Sep 06 04:21:11.542617 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 04:21:11.542617 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 04:21:11.542617 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 04:21:11.542617 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.creator() Sep 06 04:21:11.542617 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 04:21:11.542617 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application created_value = creator( Sep 06 04:21:11.544003 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 06 04:21:11.544003 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 06 04:21:11.544003 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 06 04:21:11.544003 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 06 04:21:11.544003 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 06 04:21:11.544003 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 06 04:21:11.544003 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 04:21:11.544003 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: ERROR keystone.server.flask.application Sep 06 04:21:11.544003 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 634/1285] 10.222.0.30 () {56 vars in 1111 bytes} [Tue Sep 6 04:21:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 04:21:11.549181 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b2cf28-a985-4d68-8dec-26654eda8265 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:21:11.601135 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:21:11.652047 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b2cf28-a985-4d68-8dec-26654eda8265 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:21:11.685389 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af07bfa-268f-49bb-b574-4354b160cfbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:21:11.685985 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af07bfa-268f-49bb-b574-4354b160cfbe None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:21:11.686291 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af07bfa-268f-49bb-b574-4354b160cfbe None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:21:11.686561 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af07bfa-268f-49bb-b574-4354b160cfbe None admin] PATH_INFO: `/v3/projects` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:21:11.687254 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af07bfa-268f-49bb-b574-4354b160cfbe None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:21:11.687553 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af07bfa-268f-49bb-b574-4354b160cfbe None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:21:11.689697 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af07bfa-268f-49bb-b574-4354b160cfbe None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:21:11.701407 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 652/1286] 10.222.0.30 () {56 vars in 1148 bytes} [Tue Sep 6 04:21:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:21:11.707664 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308eb35a-82e2-4f85-b392-9dab1b7ebd56 None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:21:11.806755 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308eb35a-82e2-4f85-b392-9dab1b7ebd56 None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:21:11.838481 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a07f423-e8f0-4a51-8ff6-87175ffc1637 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:21:11.839077 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a07f423-e8f0-4a51-8ff6-87175ffc1637 None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:21:11.839353 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a07f423-e8f0-4a51-8ff6-87175ffc1637 None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:21:11.839619 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a07f423-e8f0-4a51-8ff6-87175ffc1637 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:21:11.840328 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a07f423-e8f0-4a51-8ff6-87175ffc1637 None admin] RBAC: Adding query filter params (role.id=ea4c814713b149108e488de855b025e4, scope.project.id=b707733ce65c42edbc1fd95ad0a1c06c, user.id=1d4f2e4dbd41441d84f03ddb30191e7a) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:21:11.840626 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a07f423-e8f0-4a51-8ff6-87175ffc1637 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:21:11.842074 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a07f423-e8f0-4a51-8ff6-87175ffc1637 None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:21:11.848285 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 635/1287] 10.222.0.30 () {56 vars in 1374 bytes} [Tue Sep 6 04:21:11 2022] GET /identity/v3/role_assignments?user.id=1d4f2e4dbd41441d84f03ddb30191e7a&scope.project.id=b707733ce65c42edbc1fd95ad0a1c06c&role.id=ea4c814713b149108e488de855b025e4 => generated 608 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:21:13.821857 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10b895a-01bb-4ffa-ad25-2f0abb833cb2 None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:21:13.826227 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10b895a-01bb-4ffa-ad25-2f0abb833cb2 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:21:13.826227 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10b895a-01bb-4ffa-ad25-2f0abb833cb2 None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:21:13.826227 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 653/1288] 10.222.0.30 () {54 vars in 927 bytes} [Tue Sep 6 04:21:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 04:21:13.829441 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d460f0-e1a9-4ce2-88bd-c5fb3b768200 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:21:13.829875 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d460f0-e1a9-4ce2-88bd-c5fb3b768200 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:21:13.830000 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d460f0-e1a9-4ce2-88bd-c5fb3b768200 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:21:13.848652 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-47d460f0-e1a9-4ce2-88bd-c5fb3b768200 None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:21:13.849878 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-47d460f0-e1a9-4ce2-88bd-c5fb3b768200 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:21:13.878536 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:21:13.915165 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 636/1289] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Sep 6 04:21:13 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 06 04:21:14.190962 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10b895a-01bb-4ffa-ad25-2f0abb833cb2 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:21:14.293075 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10b895a-01bb-4ffa-ad25-2f0abb833cb2 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:21:14.303512 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:21:14.326653 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ed9647-c980-4de5-8bc0-d7985084f999 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:21:14.327278 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ed9647-c980-4de5-8bc0-d7985084f999 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:21:14.327555 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ed9647-c980-4de5-8bc0-d7985084f999 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:21:14.327818 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ed9647-c980-4de5-8bc0-d7985084f999 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:21:14.328991 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ed9647-c980-4de5-8bc0-d7985084f999 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:21:14.330490 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ed9647-c980-4de5-8bc0-d7985084f999 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:21:14.339010 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-16ed9647-c980-4de5-8bc0-d7985084f999 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 06 04:21:14.339010 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 04:21:14.339010 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 06 04:21:14.339010 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 06 04:21:14.339010 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 06 04:21:14.339010 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 06 04:21:14.339010 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 06 04:21:14.339010 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 04:21:14.339010 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 06 04:21:14.339010 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 04:21:14.339010 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 06 04:21:14.339010 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 04:21:14.339010 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 04:21:14.339010 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 04:21:14.339010 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 04:21:14.339010 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 04:21:14.339010 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 04:21:14.340104 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 04:21:14.340104 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 04:21:14.340104 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 04:21:14.340104 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 04:21:14.340104 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 04:21:14.340104 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 04:21:14.340104 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 06 04:21:14.340104 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 06 04:21:14.340104 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 06 04:21:14.340104 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 06 04:21:14.340104 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 04:21:14.340104 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 04:21:14.340104 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 04:21:14.340104 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 04:21:14.340104 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 04:21:14.340104 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 04:21:14.340983 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 04:21:14.340983 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 04:21:14.340983 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 04:21:14.340983 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 04:21:14.340983 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 04:21:14.340983 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 04:21:14.340983 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 04:21:14.340983 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 04:21:14.340983 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 04:21:14.340983 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 04:21:14.340983 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 06 04:21:14.340983 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 04:21:14.340983 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 06 04:21:14.340983 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 04:21:14.340983 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 654/1290] 10.222.0.30 () {56 vars in 1117 bytes} [Tue Sep 6 04:21:14 2022] GET /identity/v3/services/placement => generated 90 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 04:21:14.346597 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d460f0-e1a9-4ce2-88bd-c5fb3b768200 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:21:14.448478 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d460f0-e1a9-4ce2-88bd-c5fb3b768200 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:21:14.486500 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0e9798-6648-4aa7-8a68-6e7bfa46d73d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:21:14.487180 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0e9798-6648-4aa7-8a68-6e7bfa46d73d None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:21:14.487460 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0e9798-6648-4aa7-8a68-6e7bfa46d73d None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:21:14.487725 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0e9798-6648-4aa7-8a68-6e7bfa46d73d None admin] PATH_INFO: `/v3/services` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:21:14.488461 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa0e9798-6648-4aa7-8a68-6e7bfa46d73d None admin] RBAC: Adding query filter params (name=placement) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:21:14.488766 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa0e9798-6648-4aa7-8a68-6e7bfa46d73d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:21:14.490170 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa0e9798-6648-4aa7-8a68-6e7bfa46d73d None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:21:14.495238 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 637/1291] 10.222.0.30 () {56 vars in 1116 bytes} [Tue Sep 6 04:21:14 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:21:14.502207 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ed9647-c980-4de5-8bc0-d7985084f999 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:21:14.612399 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ed9647-c980-4de5-8bc0-d7985084f999 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:21:14.649135 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae0c34e-a37a-4c94-9cf2-7476242a7db2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:21:14.649778 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae0c34e-a37a-4c94-9cf2-7476242a7db2 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:21:14.650061 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae0c34e-a37a-4c94-9cf2-7476242a7db2 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:21:14.650332 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae0c34e-a37a-4c94-9cf2-7476242a7db2 None admin] PATH_INFO: `/v3/services` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:21:14.650964 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae0c34e-a37a-4c94-9cf2-7476242a7db2 None admin] RBAC: Adding query filter params (type=placement) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:21:14.651266 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae0c34e-a37a-4c94-9cf2-7476242a7db2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:21:14.652634 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae0c34e-a37a-4c94-9cf2-7476242a7db2 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:21:14.657748 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 655/1292] 10.222.0.30 () {56 vars in 1116 bytes} [Tue Sep 6 04:21:14 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:21:16.564723 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c13356-2b1b-4eb9-9245-617123d4796c None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:21:16.565247 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c13356-2b1b-4eb9-9245-617123d4796c None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:21:16.565247 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c13356-2b1b-4eb9-9245-617123d4796c None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:21:16.566630 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 638/1293] 10.222.0.30 () {54 vars in 927 bytes} [Tue Sep 6 04:21:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 04:21:16.571964 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2853ee29-071e-45e5-a417-33d344617c9a None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:21:16.572393 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2853ee29-071e-45e5-a417-33d344617c9a None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:21:16.572524 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2853ee29-071e-45e5-a417-33d344617c9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:21:16.591172 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-2853ee29-071e-45e5-a417-33d344617c9a None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:21:16.592357 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-2853ee29-071e-45e5-a417-33d344617c9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:21:16.621705 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:21:16.660363 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 656/1294] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Sep 6 04:21:16 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 04:21:16.931231 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c13356-2b1b-4eb9-9245-617123d4796c None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:21:16.962118 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:21:17.035814 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c13356-2b1b-4eb9-9245-617123d4796c None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:21:17.070266 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbb4227-50f2-49b2-8ef0-8ebb6c11585d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:21:17.070889 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbb4227-50f2-49b2-8ef0-8ebb6c11585d None admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:21:17.071170 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbb4227-50f2-49b2-8ef0-8ebb6c11585d None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:21:17.071442 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbb4227-50f2-49b2-8ef0-8ebb6c11585d None admin] PATH_INFO: `/v3/services` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:21:17.072093 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abbb4227-50f2-49b2-8ef0-8ebb6c11585d None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:21:17.073484 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abbb4227-50f2-49b2-8ef0-8ebb6c11585d None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:21:17.117215 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 639/1295] 10.222.0.30 () {60 vars in 1141 bytes} [Tue Sep 6 04:21:16 2022] POST /identity/v3/services => generated 246 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 04:21:19.069245 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d978c7f-0bef-4bae-94ea-68ba74b1de39 None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:21:19.069823 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d978c7f-0bef-4bae-94ea-68ba74b1de39 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:21:19.069823 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d978c7f-0bef-4bae-94ea-68ba74b1de39 None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:21:19.071183 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 657/1296] 10.222.0.30 () {54 vars in 927 bytes} [Tue Sep 6 04:21:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 06 04:21:19.076544 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfa752c-aee4-45b9-a0fd-9d92f6cc93f2 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:21:19.076957 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfa752c-aee4-45b9-a0fd-9d92f6cc93f2 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:21:19.077099 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfa752c-aee4-45b9-a0fd-9d92f6cc93f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:21:19.095794 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-9cfa752c-aee4-45b9-a0fd-9d92f6cc93f2 None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:21:19.096983 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-9cfa752c-aee4-45b9-a0fd-9d92f6cc93f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:21:19.168785 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 640/1297] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Sep 6 04:21:19 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 04:21:19.439867 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d978c7f-0bef-4bae-94ea-68ba74b1de39 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:21:19.539848 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d978c7f-0bef-4bae-94ea-68ba74b1de39 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:21:19.572748 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c14653e-16d6-4b84-8d9b-a31a448b9b65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:21:19.573416 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c14653e-16d6-4b84-8d9b-a31a448b9b65 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:21:19.573679 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c14653e-16d6-4b84-8d9b-a31a448b9b65 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:21:19.573947 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c14653e-16d6-4b84-8d9b-a31a448b9b65 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:21:19.574526 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c14653e-16d6-4b84-8d9b-a31a448b9b65 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:21:19.575898 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c14653e-16d6-4b84-8d9b-a31a448b9b65 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:21:19.585206 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-0c14653e-16d6-4b84-8d9b-a31a448b9b65 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 06 04:21:19.585206 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 04:21:19.585206 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 06 04:21:19.585206 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 06 04:21:19.585206 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 06 04:21:19.585206 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 06 04:21:19.585206 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 06 04:21:19.585206 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 04:21:19.585206 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 06 04:21:19.585206 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 04:21:19.585206 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 06 04:21:19.585206 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 04:21:19.585206 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 04:21:19.585206 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 04:21:19.585206 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 04:21:19.585206 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 04:21:19.585206 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 04:21:19.586415 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 04:21:19.586415 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 04:21:19.586415 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 04:21:19.586415 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 04:21:19.586415 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 04:21:19.586415 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 04:21:19.586415 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 06 04:21:19.586415 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 06 04:21:19.586415 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 06 04:21:19.586415 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 06 04:21:19.586415 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 04:21:19.586415 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 04:21:19.586415 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 04:21:19.586415 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 04:21:19.586415 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 04:21:19.586415 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 04:21:19.587294 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 04:21:19.587294 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 04:21:19.587294 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 04:21:19.587294 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 04:21:19.587294 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 04:21:19.587294 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 04:21:19.587294 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 04:21:19.587294 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 04:21:19.587294 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 04:21:19.587294 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 04:21:19.587294 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 06 04:21:19.587294 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 04:21:19.587294 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 06 04:21:19.587294 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 04:21:19.587294 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 658/1298] 10.222.0.30 () {56 vars in 1117 bytes} [Tue Sep 6 04:21:19 2022] GET /identity/v3/services/placement => generated 90 bytes in 147 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 04:21:19.592387 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cfa752c-aee4-45b9-a0fd-9d92f6cc93f2 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:21:19.610501 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:21:19.689912 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cfa752c-aee4-45b9-a0fd-9d92f6cc93f2 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:21:19.722889 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6e7d2d-eebd-4efd-98da-38228d2a213b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:21:19.723502 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6e7d2d-eebd-4efd-98da-38228d2a213b None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:21:19.723789 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6e7d2d-eebd-4efd-98da-38228d2a213b None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:21:19.724063 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6e7d2d-eebd-4efd-98da-38228d2a213b None admin] PATH_INFO: `/v3/services` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:21:19.724693 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6e7d2d-eebd-4efd-98da-38228d2a213b None admin] RBAC: Adding query filter params (name=placement) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:21:19.724997 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6e7d2d-eebd-4efd-98da-38228d2a213b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:21:19.726399 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6e7d2d-eebd-4efd-98da-38228d2a213b None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:21:19.731453 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 641/1299] 10.222.0.30 () {56 vars in 1116 bytes} [Tue Sep 6 04:21:19 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:21:19.738096 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c14653e-16d6-4b84-8d9b-a31a448b9b65 None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:21:19.770086 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:21:19.839326 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c14653e-16d6-4b84-8d9b-a31a448b9b65 None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:21:19.872897 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2496a56-53e7-4cda-8f75-f4ca2e485670 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:21:19.873534 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2496a56-53e7-4cda-8f75-f4ca2e485670 None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:21:19.873817 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2496a56-53e7-4cda-8f75-f4ca2e485670 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:21:19.874089 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2496a56-53e7-4cda-8f75-f4ca2e485670 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:21:19.874790 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2496a56-53e7-4cda-8f75-f4ca2e485670 None admin] RBAC: Adding query filter params (interface=public, service_id=c568c4caf006441c853636b69479cd5f, region_id=RegionOne) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:21:19.875699 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2496a56-53e7-4cda-8f75-f4ca2e485670 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:21:19.877339 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2496a56-53e7-4cda-8f75-f4ca2e485670 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:21:19.882361 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 659/1300] 10.222.0.30 () {56 vars in 1251 bytes} [Tue Sep 6 04:21:19 2022] GET /identity/v3/endpoints?service_id=c568c4caf006441c853636b69479cd5f&interface=public®ion_id=RegionOne => generated 194 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:21:19.888126 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6e7d2d-eebd-4efd-98da-38228d2a213b None admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:21:19.976359 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:21:19.994041 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6e7d2d-eebd-4efd-98da-38228d2a213b None admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:21:20.029235 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562e07d1-24f0-4871-a69d-941ad1f468dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:21:20.029881 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562e07d1-24f0-4871-a69d-941ad1f468dd None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:21:20.030160 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562e07d1-24f0-4871-a69d-941ad1f468dd None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:21:20.030434 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562e07d1-24f0-4871-a69d-941ad1f468dd None admin] PATH_INFO: `/v3/services` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:21:20.031045 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562e07d1-24f0-4871-a69d-941ad1f468dd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:21:20.032840 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562e07d1-24f0-4871-a69d-941ad1f468dd None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:21:20.039648 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 642/1301] 10.222.0.30 () {56 vars in 1087 bytes} [Tue Sep 6 04:21:19 2022] GET /identity/v3/services => generated 2191 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 04:21:21.957775 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211bbf13-d713-4bab-9a17-9178d5e29865 None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:21:21.959006 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211bbf13-d713-4bab-9a17-9178d5e29865 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:21:21.959296 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211bbf13-d713-4bab-9a17-9178d5e29865 None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:21:21.960664 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 660/1302] 10.222.0.30 () {54 vars in 927 bytes} [Tue Sep 6 04:21:21 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 04:21:21.966097 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec000de8-1c2a-4a92-aadd-af171fb8eb7a None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:21:21.966510 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec000de8-1c2a-4a92-aadd-af171fb8eb7a None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:21:21.966646 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec000de8-1c2a-4a92-aadd-af171fb8eb7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:21:21.985054 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-ec000de8-1c2a-4a92-aadd-af171fb8eb7a None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:21:21.986085 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-ec000de8-1c2a-4a92-aadd-af171fb8eb7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:21:22.052996 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 643/1303] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Sep 6 04:21:21 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 04:21:22.322837 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211bbf13-d713-4bab-9a17-9178d5e29865 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:21:22.417231 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:21:22.424317 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211bbf13-d713-4bab-9a17-9178d5e29865 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:21:22.457655 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc5200f-2fa4-4d36-8ff9-1af035ea331e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:21:22.458281 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc5200f-2fa4-4d36-8ff9-1af035ea331e None admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:21:22.458560 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc5200f-2fa4-4d36-8ff9-1af035ea331e None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:21:22.458836 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc5200f-2fa4-4d36-8ff9-1af035ea331e None admin] PATH_INFO: `/v3/services/placement` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:21:22.459416 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc5200f-2fa4-4d36-8ff9-1af035ea331e None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:21:22.460793 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc5200f-2fa4-4d36-8ff9-1af035ea331e None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:21:22.469857 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application [None req-4cc5200f-2fa4-4d36-8ff9-1af035ea331e None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 06 04:21:22.469857 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 04:21:22.469857 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 06 04:21:22.469857 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 06 04:21:22.469857 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 06 04:21:22.469857 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 06 04:21:22.469857 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 06 04:21:22.469857 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 04:21:22.469857 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 06 04:21:22.469857 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 04:21:22.469857 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 06 04:21:22.469857 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 04:21:22.469857 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 04:21:22.469857 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 04:21:22.469857 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 04:21:22.469857 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 04:21:22.469857 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 04:21:22.471121 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 04:21:22.471121 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 04:21:22.471121 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 04:21:22.471121 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 04:21:22.471121 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 04:21:22.471121 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 04:21:22.471121 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 06 04:21:22.471121 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 06 04:21:22.471121 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 06 04:21:22.471121 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 06 04:21:22.471121 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 04:21:22.471121 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 04:21:22.471121 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 04:21:22.471121 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 04:21:22.471121 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 04:21:22.471121 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 04:21:22.472165 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 04:21:22.472165 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application with Lock( Sep 06 04:21:22.472165 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 04:21:22.472165 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self._enter() Sep 06 04:21:22.472165 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 04:21:22.472165 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 04:21:22.472165 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 04:21:22.472165 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application return self.creator() Sep 06 04:21:22.472165 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 04:21:22.472165 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application created_value = creator( Sep 06 04:21:22.472165 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 06 04:21:22.472165 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 04:21:22.472165 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 06 04:21:22.472165 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: ERROR keystone.server.flask.application Sep 06 04:21:22.472165 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 661/1304] 10.222.0.30 () {56 vars in 1117 bytes} [Tue Sep 6 04:21:22 2022] GET /identity/v3/services/placement => generated 90 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 04:21:22.476985 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec000de8-1c2a-4a92-aadd-af171fb8eb7a None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:21:22.553369 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:21:22.575756 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec000de8-1c2a-4a92-aadd-af171fb8eb7a None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:21:22.608641 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c543c003-d7eb-43f3-801f-28038b5c714b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:21:22.609245 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c543c003-d7eb-43f3-801f-28038b5c714b None admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:21:22.609722 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c543c003-d7eb-43f3-801f-28038b5c714b None admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:21:22.610148 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c543c003-d7eb-43f3-801f-28038b5c714b None admin] PATH_INFO: `/v3/services` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:21:22.610780 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c543c003-d7eb-43f3-801f-28038b5c714b None admin] RBAC: Adding query filter params (name=placement) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:21:22.611089 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c543c003-d7eb-43f3-801f-28038b5c714b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:21:22.612457 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c543c003-d7eb-43f3-801f-28038b5c714b None admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:21:22.617506 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 644/1305] 10.222.0.30 () {56 vars in 1116 bytes} [Tue Sep 6 04:21:22 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 04:21:22.623345 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc5200f-2fa4-4d36-8ff9-1af035ea331e None admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:21:22.727532 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc5200f-2fa4-4d36-8ff9-1af035ea331e None admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:21:22.762290 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0194aad7-1bd3-4d02-bcd9-e642d3da7770 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:21:22.762898 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0194aad7-1bd3-4d02-bcd9-e642d3da7770 None admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:21:22.763176 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0194aad7-1bd3-4d02-bcd9-e642d3da7770 None admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:21:22.763445 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0194aad7-1bd3-4d02-bcd9-e642d3da7770 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:21:22.764118 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0194aad7-1bd3-4d02-bcd9-e642d3da7770 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:21:22.765708 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0194aad7-1bd3-4d02-bcd9-e642d3da7770 None admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:21:22.832080 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 662/1306] 10.222.0.30 () {60 vars in 1144 bytes} [Tue Sep 6 04:21:22 2022] POST /identity/v3/endpoints => generated 331 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 04:21:32.678849 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=30 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:21:32.827411 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:27:15.346516 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda1ff51-4b46-4268-a875-850174192b2b None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:27:15.347686 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda1ff51-4b46-4268-a875-850174192b2b None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:27:15.348254 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda1ff51-4b46-4268-a875-850174192b2b None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:27:15.350292 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 645/1307] 10.222.0.30 () {54 vars in 927 bytes} [Tue Sep 6 04:27:15 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 04:27:15.355341 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bc1b43-a7e8-4f90-a3d9-7b138ecb4cf7 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:27:15.355794 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bc1b43-a7e8-4f90-a3d9-7b138ecb4cf7 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:27:15.355898 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bc1b43-a7e8-4f90-a3d9-7b138ecb4cf7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:27:15.432886 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-93bc1b43-a7e8-4f90-a3d9-7b138ecb4cf7 None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:27:15.434141 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-93bc1b43-a7e8-4f90-a3d9-7b138ecb4cf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:27:15.642895 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 663/1308] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Sep 6 04:27:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 288 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 04:27:15.945722 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99b07e1-d9a6-40b8-879a-5f36ccfdbd4a None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:27:15.946352 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99b07e1-d9a6-40b8-879a-5f36ccfdbd4a None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:27:15.946871 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99b07e1-d9a6-40b8-879a-5f36ccfdbd4a None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:27:15.948696 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 646/1309] 10.222.0.30 () {54 vars in 965 bytes} [Tue Sep 6 04:27:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 04:27:15.954785 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ae4f79-025b-4376-9ebf-f99ebb82c605 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:27:15.955079 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ae4f79-025b-4376-9ebf-f99ebb82c605 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:27:15.955358 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ae4f79-025b-4376-9ebf-f99ebb82c605 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:27:16.057515 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-e8ae4f79-025b-4376-9ebf-f99ebb82c605 None None] MFA Rules not processed for user `90492440f6a6471e8392361c098428aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:27:16.059265 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-e8ae4f79-025b-4376-9ebf-f99ebb82c605 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:27:16.087510 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:27:16.157209 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 664/1310] 10.222.0.30 () {58 vars in 1062 bytes} [Tue Sep 6 04:27:15 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 04:27:16.166537 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99b07e1-d9a6-40b8-879a-5f36ccfdbd4a None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:27:16.168604 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-f99b07e1-d9a6-40b8-879a-5f36ccfdbd4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:27:16.300251 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99b07e1-d9a6-40b8-879a-5f36ccfdbd4a None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:27:16.387608 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421c7c03-bff3-4cd4-a767-3bd58772f484 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90492440f6a6471e8392361c098428aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b707733ce65c42edbc1fd95ad0a1c06c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:27:16.388366 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421c7c03-bff3-4cd4-a767-3bd58772f484 service neutron] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:27:16.388655 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421c7c03-bff3-4cd4-a767-3bd58772f484 service neutron] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:27:16.388931 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421c7c03-bff3-4cd4-a767-3bd58772f484 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:27:16.390896 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-421c7c03-bff3-4cd4-a767-3bd58772f484 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:27:16.449906 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:27:16.509524 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421c7c03-bff3-4cd4-a767-3bd58772f484 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:27:16.511091 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421c7c03-bff3-4cd4-a767-3bd58772f484 service neutron] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:27:16.532226 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 647/1311] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Sep 6 04:27:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 04:27:18.756351 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9397327-b8cc-4f46-ac29-1485184ae0f9 None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:27:18.756893 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9397327-b8cc-4f46-ac29-1485184ae0f9 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:27:18.756893 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9397327-b8cc-4f46-ac29-1485184ae0f9 None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:27:18.758418 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 665/1312] 10.222.0.30 () {54 vars in 927 bytes} [Tue Sep 6 04:27:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 06 04:27:18.763368 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa063b4-6bbb-4a04-a60a-91807af0192b None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:27:18.763747 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa063b4-6bbb-4a04-a60a-91807af0192b None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:27:18.763968 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa063b4-6bbb-4a04-a60a-91807af0192b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:27:18.849832 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-0fa063b4-6bbb-4a04-a60a-91807af0192b None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:27:18.851031 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-0fa063b4-6bbb-4a04-a60a-91807af0192b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:27:18.951556 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 648/1313] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Sep 6 04:27:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 04:27:19.236678 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9397327-b8cc-4f46-ac29-1485184ae0f9 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:27:19.238266 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-c9397327-b8cc-4f46-ac29-1485184ae0f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:27:19.346104 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9397327-b8cc-4f46-ac29-1485184ae0f9 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:27:19.426927 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:27:19.429629 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4dbf06-0a64-4a62-b13b-dc18277df322 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90492440f6a6471e8392361c098428aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b707733ce65c42edbc1fd95ad0a1c06c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:27:19.430250 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4dbf06-0a64-4a62-b13b-dc18277df322 service neutron] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:27:19.430531 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4dbf06-0a64-4a62-b13b-dc18277df322 service neutron] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:27:19.430803 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4dbf06-0a64-4a62-b13b-dc18277df322 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:27:19.432413 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-9e4dbf06-0a64-4a62-b13b-dc18277df322 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:27:19.545200 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4dbf06-0a64-4a62-b13b-dc18277df322 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:27:19.546596 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4dbf06-0a64-4a62-b13b-dc18277df322 service neutron] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:27:19.565790 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 666/1314] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Sep 6 04:27:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 04:27:22.010564 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647be7b8-7ead-4f4f-8b53-a4a4fe341d80 None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:27:22.011125 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647be7b8-7ead-4f4f-8b53-a4a4fe341d80 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:27:22.011125 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647be7b8-7ead-4f4f-8b53-a4a4fe341d80 None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:27:22.012509 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 649/1315] 10.222.0.30 () {54 vars in 927 bytes} [Tue Sep 6 04:27:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 04:27:22.017814 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecba7be8-2f69-4c43-abd3-fe136221f1f0 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:27:22.018216 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecba7be8-2f69-4c43-abd3-fe136221f1f0 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:27:22.018354 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecba7be8-2f69-4c43-abd3-fe136221f1f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:27:22.089019 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-ecba7be8-2f69-4c43-abd3-fe136221f1f0 None None] MFA Rules not processed for user `46380000cf7c43d197c6d8207cc91e2a`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:27:22.090288 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-ecba7be8-2f69-4c43-abd3-fe136221f1f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:27:22.142575 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:27:22.190505 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 667/1316] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Sep 6 04:27:22 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 04:27:22.474035 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647be7b8-7ead-4f4f-8b53-a4a4fe341d80 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:27:22.475937 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-647be7b8-7ead-4f4f-8b53-a4a4fe341d80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:27:22.485563 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:27:22.578395 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647be7b8-7ead-4f4f-8b53-a4a4fe341d80 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:27:22.650643 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1062e1-78dd-4025-b978-e6a0c1512db7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90492440f6a6471e8392361c098428aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b707733ce65c42edbc1fd95ad0a1c06c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:27:22.651185 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1062e1-78dd-4025-b978-e6a0c1512db7 service neutron] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:27:22.651460 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1062e1-78dd-4025-b978-e6a0c1512db7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:27:22.651727 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1062e1-78dd-4025-b978-e6a0c1512db7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:27:22.653294 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-4b1062e1-78dd-4025-b978-e6a0c1512db7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:27:22.713956 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:27:22.758798 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1062e1-78dd-4025-b978-e6a0c1512db7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:27:22.760217 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1062e1-78dd-4025-b978-e6a0c1512db7 service neutron] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:27:22.778367 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 650/1317] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Sep 6 04:27:22 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 04:27:25.482000 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886a1efd-1a28-4835-afc8-9d29bc44a582 None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:27:25.482513 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886a1efd-1a28-4835-afc8-9d29bc44a582 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:27:25.482607 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886a1efd-1a28-4835-afc8-9d29bc44a582 None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:27:25.483900 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 668/1318] 10.222.0.30 () {54 vars in 927 bytes} [Tue Sep 6 04:27:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 04:27:25.489076 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd53fa9-5fcd-4642-8645-e72399416c1f None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:27:25.489497 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd53fa9-5fcd-4642-8645-e72399416c1f None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:27:25.489663 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd53fa9-5fcd-4642-8645-e72399416c1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:27:25.554667 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-2cd53fa9-5fcd-4642-8645-e72399416c1f None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:27:25.555877 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-2cd53fa9-5fcd-4642-8645-e72399416c1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:27:25.651105 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 651/1319] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Sep 6 04:27:25 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 04:27:25.943344 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886a1efd-1a28-4835-afc8-9d29bc44a582 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:27:25.945138 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-886a1efd-1a28-4835-afc8-9d29bc44a582 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:27:26.049323 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886a1efd-1a28-4835-afc8-9d29bc44a582 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:27:26.124425 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ec3d59-0a7e-4a71-b0e7-0646852ad26d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90492440f6a6471e8392361c098428aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b707733ce65c42edbc1fd95ad0a1c06c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:27:26.125024 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ec3d59-0a7e-4a71-b0e7-0646852ad26d service neutron] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:27:26.125297 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ec3d59-0a7e-4a71-b0e7-0646852ad26d service neutron] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:27:26.125592 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ec3d59-0a7e-4a71-b0e7-0646852ad26d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:27:26.127171 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-68ec3d59-0a7e-4a71-b0e7-0646852ad26d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:27:26.134476 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:27:26.236103 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ec3d59-0a7e-4a71-b0e7-0646852ad26d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:27:26.237502 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ec3d59-0a7e-4a71-b0e7-0646852ad26d service neutron] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:27:26.256037 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 669/1320] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Sep 6 04:27:25 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 06 04:27:29.206819 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394dcaa5-64b7-4b98-a75a-1ad10059d76d None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:27:29.207532 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394dcaa5-64b7-4b98-a75a-1ad10059d76d None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:27:29.207532 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394dcaa5-64b7-4b98-a75a-1ad10059d76d None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:27:29.208833 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 652/1321] 10.222.0.30 () {54 vars in 927 bytes} [Tue Sep 6 04:27:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 04:27:29.214530 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924161f9-c5c1-465b-877c-d9a2f09c41e7 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:27:29.214986 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924161f9-c5c1-465b-877c-d9a2f09c41e7 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:27:29.215080 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924161f9-c5c1-465b-877c-d9a2f09c41e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:27:29.284584 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-924161f9-c5c1-465b-877c-d9a2f09c41e7 None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:27:29.286440 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-924161f9-c5c1-465b-877c-d9a2f09c41e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:27:29.343360 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:27:29.385511 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 670/1322] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Sep 6 04:27:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 04:27:29.675663 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394dcaa5-64b7-4b98-a75a-1ad10059d76d None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:27:29.677412 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-394dcaa5-64b7-4b98-a75a-1ad10059d76d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:27:29.706089 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:27:29.795621 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394dcaa5-64b7-4b98-a75a-1ad10059d76d None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:27:29.870057 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2057861d-b9a8-49a5-89ac-1b049c1663b6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90492440f6a6471e8392361c098428aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b707733ce65c42edbc1fd95ad0a1c06c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:27:29.870673 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2057861d-b9a8-49a5-89ac-1b049c1663b6 service neutron] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:27:29.870950 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2057861d-b9a8-49a5-89ac-1b049c1663b6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:27:29.871219 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2057861d-b9a8-49a5-89ac-1b049c1663b6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:27:29.872802 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-2057861d-b9a8-49a5-89ac-1b049c1663b6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:27:29.944753 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:27:29.982438 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2057861d-b9a8-49a5-89ac-1b049c1663b6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:27:29.983803 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2057861d-b9a8-49a5-89ac-1b049c1663b6 service neutron] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:27:30.002060 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 653/1323] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Sep 6 04:27:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 06 04:27:31.968002 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4099ed14-9cc4-4357-b2fc-1a1dee0f09be None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:27:31.968527 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4099ed14-9cc4-4357-b2fc-1a1dee0f09be None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:27:31.968527 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4099ed14-9cc4-4357-b2fc-1a1dee0f09be None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:27:31.969951 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 671/1324] 10.222.0.30 () {54 vars in 927 bytes} [Tue Sep 6 04:27:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 04:27:31.975828 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d95b110-97fe-4200-9a04-4fff8392804e None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:27:31.976247 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d95b110-97fe-4200-9a04-4fff8392804e None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:27:31.976537 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d95b110-97fe-4200-9a04-4fff8392804e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:27:32.043247 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-7d95b110-97fe-4200-9a04-4fff8392804e None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:27:32.044383 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-7d95b110-97fe-4200-9a04-4fff8392804e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:27:32.192678 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 654/1325] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Sep 6 04:27:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 04:27:32.440200 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4099ed14-9cc4-4357-b2fc-1a1dee0f09be None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:27:32.442388 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-4099ed14-9cc4-4357-b2fc-1a1dee0f09be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:27:32.552884 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4099ed14-9cc4-4357-b2fc-1a1dee0f09be None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:27:32.628477 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5012177-9c61-4fe0-873d-15ab0b95eb87 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90492440f6a6471e8392361c098428aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b707733ce65c42edbc1fd95ad0a1c06c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:27:32.629710 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5012177-9c61-4fe0-873d-15ab0b95eb87 service neutron] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:27:32.629984 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5012177-9c61-4fe0-873d-15ab0b95eb87 service neutron] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:27:32.630249 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5012177-9c61-4fe0-873d-15ab0b95eb87 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:27:32.631953 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-a5012177-9c61-4fe0-873d-15ab0b95eb87 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:27:32.649638 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:27:32.751552 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5012177-9c61-4fe0-873d-15ab0b95eb87 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:27:32.753248 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5012177-9c61-4fe0-873d-15ab0b95eb87 service neutron] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:27:32.772807 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 672/1326] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Sep 6 04:27:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 06 04:27:34.753845 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848ebefa-e8bd-4a3c-8341-5b7e598627ea None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:27:34.754420 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848ebefa-e8bd-4a3c-8341-5b7e598627ea None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:27:34.754420 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848ebefa-e8bd-4a3c-8341-5b7e598627ea None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:27:34.756216 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 655/1327] 10.222.0.30 () {54 vars in 927 bytes} [Tue Sep 6 04:27:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 04:27:34.761248 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9099d264-958c-4b32-b434-ebf649964c5d None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:27:34.761702 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9099d264-958c-4b32-b434-ebf649964c5d None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:27:34.761947 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9099d264-958c-4b32-b434-ebf649964c5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:27:34.830701 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-9099d264-958c-4b32-b434-ebf649964c5d None None] MFA Rules not processed for user `46380000cf7c43d197c6d8207cc91e2a`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:27:34.831905 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-9099d264-958c-4b32-b434-ebf649964c5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:27:34.903566 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:27:34.943455 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 673/1328] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Sep 6 04:27:34 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 04:27:35.225455 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848ebefa-e8bd-4a3c-8341-5b7e598627ea None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:27:35.226990 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-848ebefa-e8bd-4a3c-8341-5b7e598627ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:27:35.258127 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:27:35.344142 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848ebefa-e8bd-4a3c-8341-5b7e598627ea None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:27:35.426998 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a9c886-d8cf-4188-bc4b-ae49d46256a9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90492440f6a6471e8392361c098428aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b707733ce65c42edbc1fd95ad0a1c06c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:27:35.427619 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a9c886-d8cf-4188-bc4b-ae49d46256a9 service neutron] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:27:35.427903 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a9c886-d8cf-4188-bc4b-ae49d46256a9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:27:35.428175 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a9c886-d8cf-4188-bc4b-ae49d46256a9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:27:35.429829 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-90a9c886-d8cf-4188-bc4b-ae49d46256a9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:27:35.509498 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:27:35.541294 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a9c886-d8cf-4188-bc4b-ae49d46256a9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:27:35.542699 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a9c886-d8cf-4188-bc4b-ae49d46256a9 service neutron] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:27:35.561196 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 656/1329] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Sep 6 04:27:35 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 04:27:37.906408 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4e08b4-f4e7-40c7-be6e-edecb0150dcc None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:27:37.906988 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4e08b4-f4e7-40c7-be6e-edecb0150dcc None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:27:37.906988 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4e08b4-f4e7-40c7-be6e-edecb0150dcc None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:27:37.908397 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 674/1330] 10.222.0.30 () {54 vars in 927 bytes} [Tue Sep 6 04:27:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 04:27:37.914431 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0cb9543-2f42-47e4-bc61-16c8c7b6f522 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:27:37.914968 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0cb9543-2f42-47e4-bc61-16c8c7b6f522 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:27:37.914968 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0cb9543-2f42-47e4-bc61-16c8c7b6f522 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:27:37.980312 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-f0cb9543-2f42-47e4-bc61-16c8c7b6f522 None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:27:37.981559 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-f0cb9543-2f42-47e4-bc61-16c8c7b6f522 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:27:38.077802 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 657/1331] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Sep 6 04:27:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 04:27:38.359469 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4e08b4-f4e7-40c7-be6e-edecb0150dcc None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:27:38.361020 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-ab4e08b4-f4e7-40c7-be6e-edecb0150dcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:27:38.466338 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4e08b4-f4e7-40c7-be6e-edecb0150dcc None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:27:38.540176 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef6e596-0391-4bba-9897-4874ac106c8e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90492440f6a6471e8392361c098428aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b707733ce65c42edbc1fd95ad0a1c06c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:27:38.540784 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef6e596-0391-4bba-9897-4874ac106c8e service neutron] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:27:38.541057 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef6e596-0391-4bba-9897-4874ac106c8e service neutron] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:27:38.541348 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef6e596-0391-4bba-9897-4874ac106c8e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:27:38.542946 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-bef6e596-0391-4bba-9897-4874ac106c8e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:27:38.568683 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:27:38.651124 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef6e596-0391-4bba-9897-4874ac106c8e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:27:38.652472 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef6e596-0391-4bba-9897-4874ac106c8e service neutron] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:27:38.674723 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 675/1332] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Sep 6 04:27:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 04:27:40.688208 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c9b203-2112-4b3d-9c69-92c67bed6fae None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:27:40.688723 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c9b203-2112-4b3d-9c69-92c67bed6fae None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:27:40.688723 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c9b203-2112-4b3d-9c69-92c67bed6fae None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:27:40.690060 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 658/1333] 10.222.0.30 () {54 vars in 927 bytes} [Tue Sep 6 04:27:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 04:27:40.695970 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72de3a11-69ca-4f8f-8cee-848a96081b4a None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:27:40.696388 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72de3a11-69ca-4f8f-8cee-848a96081b4a None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:27:40.696509 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72de3a11-69ca-4f8f-8cee-848a96081b4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:27:40.761419 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-72de3a11-69ca-4f8f-8cee-848a96081b4a None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:27:40.762621 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-72de3a11-69ca-4f8f-8cee-848a96081b4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:27:40.836816 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:27:40.859205 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 676/1334] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Sep 6 04:27:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 04:27:41.149788 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c9b203-2112-4b3d-9c69-92c67bed6fae None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:27:41.151354 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-33c9b203-2112-4b3d-9c69-92c67bed6fae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:27:41.190010 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:27:41.252213 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c9b203-2112-4b3d-9c69-92c67bed6fae None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:27:41.328171 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f08219e-5291-4f2b-bef3-d61db5836d9f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90492440f6a6471e8392361c098428aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b707733ce65c42edbc1fd95ad0a1c06c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:27:41.328791 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f08219e-5291-4f2b-bef3-d61db5836d9f service neutron] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:27:41.329071 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f08219e-5291-4f2b-bef3-d61db5836d9f service neutron] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:27:41.329365 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f08219e-5291-4f2b-bef3-d61db5836d9f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:27:41.330991 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-7f08219e-5291-4f2b-bef3-d61db5836d9f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:27:41.417034 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:27:41.443708 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f08219e-5291-4f2b-bef3-d61db5836d9f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:27:41.445162 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f08219e-5291-4f2b-bef3-d61db5836d9f service neutron] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:27:41.463153 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 659/1335] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Sep 6 04:27:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 04:27:43.993578 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1a181f-9f6f-4c24-b112-8ed5f0b96ae3 None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:27:43.994179 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1a181f-9f6f-4c24-b112-8ed5f0b96ae3 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:27:43.994179 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1a181f-9f6f-4c24-b112-8ed5f0b96ae3 None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:27:43.995471 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 677/1336] 10.222.0.30 () {54 vars in 927 bytes} [Tue Sep 6 04:27:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 04:27:44.000492 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e98eee0-30ac-465e-a48a-330fab309894 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:27:44.000885 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e98eee0-30ac-465e-a48a-330fab309894 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:27:44.001038 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e98eee0-30ac-465e-a48a-330fab309894 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:27:44.069126 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-2e98eee0-30ac-465e-a48a-330fab309894 None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:27:44.070375 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-2e98eee0-30ac-465e-a48a-330fab309894 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:27:44.166444 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 660/1337] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Sep 6 04:27:43 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 04:27:44.445112 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1a181f-9f6f-4c24-b112-8ed5f0b96ae3 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:27:44.446682 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-1a1a181f-9f6f-4c24-b112-8ed5f0b96ae3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:27:44.548909 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1a181f-9f6f-4c24-b112-8ed5f0b96ae3 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:27:44.621879 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fcaae16-7d0e-4d88-add0-6f76ef4e8cce service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90492440f6a6471e8392361c098428aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b707733ce65c42edbc1fd95ad0a1c06c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:27:44.622485 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcaae16-7d0e-4d88-add0-6f76ef4e8cce service neutron] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:27:44.623219 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcaae16-7d0e-4d88-add0-6f76ef4e8cce service neutron] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:27:44.623494 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcaae16-7d0e-4d88-add0-6f76ef4e8cce service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:27:44.625077 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-4fcaae16-7d0e-4d88-add0-6f76ef4e8cce service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:27:44.659607 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:27:44.730927 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fcaae16-7d0e-4d88-add0-6f76ef4e8cce service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:27:44.732362 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fcaae16-7d0e-4d88-add0-6f76ef4e8cce service neutron] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:27:44.751386 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 678/1338] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Sep 6 04:27:44 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 04:27:49.076068 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ecb78d-89e4-4910-b400-1d590f99f311 None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:27:49.076624 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ecb78d-89e4-4910-b400-1d590f99f311 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:27:49.076769 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ecb78d-89e4-4910-b400-1d590f99f311 None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:27:49.078320 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 661/1339] 10.222.0.30 () {54 vars in 927 bytes} [Tue Sep 6 04:27:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 04:27:49.084013 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330baa42-8677-48cd-8731-45f6258e7815 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:27:49.084450 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330baa42-8677-48cd-8731-45f6258e7815 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:27:49.084568 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330baa42-8677-48cd-8731-45f6258e7815 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:27:49.154867 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-330baa42-8677-48cd-8731-45f6258e7815 None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:27:49.156019 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-330baa42-8677-48cd-8731-45f6258e7815 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:27:49.246404 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:27:49.257657 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 679/1340] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Sep 6 04:27:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 04:27:49.557870 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ea28ea-b9db-4087-b35e-3669f9220869 None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:27:49.558158 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ea28ea-b9db-4087-b35e-3669f9220869 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:27:49.558439 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ea28ea-b9db-4087-b35e-3669f9220869 None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:27:49.559828 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 662/1341] 10.222.0.30 () {54 vars in 965 bytes} [Tue Sep 6 04:27:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 04:27:49.565941 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6138a6-2c75-4353-8bfe-66f202215d9b None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:27:49.566228 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6138a6-2c75-4353-8bfe-66f202215d9b None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:27:49.567120 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6138a6-2c75-4353-8bfe-66f202215d9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:27:49.643663 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-cc6138a6-2c75-4353-8bfe-66f202215d9b None None] MFA Rules not processed for user `90492440f6a6471e8392361c098428aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:27:49.644953 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-cc6138a6-2c75-4353-8bfe-66f202215d9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:27:49.720065 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 680/1342] 10.222.0.30 () {58 vars in 1062 bytes} [Tue Sep 6 04:27:49 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 04:27:49.729296 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ea28ea-b9db-4087-b35e-3669f9220869 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:27:49.730883 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-53ea28ea-b9db-4087-b35e-3669f9220869 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:27:49.788531 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:27:49.843430 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ea28ea-b9db-4087-b35e-3669f9220869 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:27:49.919433 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f233830-71da-469d-8baa-ea19a313951a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90492440f6a6471e8392361c098428aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b707733ce65c42edbc1fd95ad0a1c06c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:27:49.920044 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f233830-71da-469d-8baa-ea19a313951a service neutron] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:27:49.920323 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f233830-71da-469d-8baa-ea19a313951a service neutron] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:27:49.920596 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f233830-71da-469d-8baa-ea19a313951a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:27:49.922218 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-3f233830-71da-469d-8baa-ea19a313951a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:27:50.017095 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:27:50.033730 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f233830-71da-469d-8baa-ea19a313951a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:27:50.035199 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f233830-71da-469d-8baa-ea19a313951a service neutron] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:27:50.053850 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 663/1343] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Sep 6 04:27:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 04:27:53.577024 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c76114-dd45-42f2-8e92-b7eb217e65ea None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:27:53.577517 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c76114-dd45-42f2-8e92-b7eb217e65ea None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:27:53.577647 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c76114-dd45-42f2-8e92-b7eb217e65ea None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:27:53.579047 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 681/1344] 10.222.0.30 () {54 vars in 927 bytes} [Tue Sep 6 04:27:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 04:27:53.584320 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de31e9ed-224c-482a-a097-12a32a2d95c7 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:27:53.584747 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de31e9ed-224c-482a-a097-12a32a2d95c7 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:27:53.584863 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de31e9ed-224c-482a-a097-12a32a2d95c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:27:53.657331 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-de31e9ed-224c-482a-a097-12a32a2d95c7 None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:27:53.658573 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-de31e9ed-224c-482a-a097-12a32a2d95c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:27:53.757551 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 664/1345] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Sep 6 04:27:53 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 04:27:54.041259 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c76114-dd45-42f2-8e92-b7eb217e65ea None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:27:54.042820 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-a7c76114-dd45-42f2-8e92-b7eb217e65ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:27:54.146020 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:27:54.164024 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c76114-dd45-42f2-8e92-b7eb217e65ea None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:27:54.243528 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf05d717-4358-4197-a4d2-576bc56e65fa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90492440f6a6471e8392361c098428aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b707733ce65c42edbc1fd95ad0a1c06c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:27:54.244136 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf05d717-4358-4197-a4d2-576bc56e65fa service neutron] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:27:54.244410 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf05d717-4358-4197-a4d2-576bc56e65fa service neutron] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:27:54.244677 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf05d717-4358-4197-a4d2-576bc56e65fa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:27:54.246314 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-cf05d717-4358-4197-a4d2-576bc56e65fa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:27:54.369384 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf05d717-4358-4197-a4d2-576bc56e65fa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:27:54.370873 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf05d717-4358-4197-a4d2-576bc56e65fa service neutron] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:27:54.389362 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 682/1346] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Sep 6 04:27:54 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 06 04:27:58.690678 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2f0daf-510f-4dba-839b-affbcf1218fe None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:27:58.691227 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2f0daf-510f-4dba-839b-affbcf1218fe None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:27:58.691227 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2f0daf-510f-4dba-839b-affbcf1218fe None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:27:58.692818 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 665/1347] 10.222.0.30 () {54 vars in 927 bytes} [Tue Sep 6 04:27:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 04:27:58.698265 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cca392d-c7af-4b80-9cbb-100e90fc8628 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:27:58.698736 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cca392d-c7af-4b80-9cbb-100e90fc8628 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:27:58.698829 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cca392d-c7af-4b80-9cbb-100e90fc8628 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:27:58.715687 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:27:58.772740 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-8cca392d-c7af-4b80-9cbb-100e90fc8628 None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:27:58.774000 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-8cca392d-c7af-4b80-9cbb-100e90fc8628 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:27:58.884967 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 683/1348] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Sep 6 04:27:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 04:27:59.178238 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2f0daf-510f-4dba-839b-affbcf1218fe None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:27:59.180774 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-1c2f0daf-510f-4dba-839b-affbcf1218fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:27:59.244000 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:27:59.291079 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2f0daf-510f-4dba-839b-affbcf1218fe None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:27:59.365613 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b8aa06-b6ca-4fb4-8a76-2f7a6abdfdb6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90492440f6a6471e8392361c098428aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b707733ce65c42edbc1fd95ad0a1c06c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:27:59.366231 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b8aa06-b6ca-4fb4-8a76-2f7a6abdfdb6 service neutron] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:27:59.366513 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b8aa06-b6ca-4fb4-8a76-2f7a6abdfdb6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:27:59.366785 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b8aa06-b6ca-4fb4-8a76-2f7a6abdfdb6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:27:59.368400 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-70b8aa06-b6ca-4fb4-8a76-2f7a6abdfdb6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:27:59.475682 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:27:59.479219 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b8aa06-b6ca-4fb4-8a76-2f7a6abdfdb6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:27:59.480683 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b8aa06-b6ca-4fb4-8a76-2f7a6abdfdb6 service neutron] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:27:59.498562 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 666/1349] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Sep 6 04:27:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 04:28:04.415514 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024d4088-1f8e-40be-acba-80e86ce254cf None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:28:04.416053 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024d4088-1f8e-40be-acba-80e86ce254cf None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:28:04.416053 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024d4088-1f8e-40be-acba-80e86ce254cf None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:28:04.418079 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 684/1350] 10.222.0.30 () {54 vars in 921 bytes} [Tue Sep 6 04:28:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 04:28:04.423059 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0388da46-37c0-4832-839a-7445e9038984 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:28:04.423493 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0388da46-37c0-4832-839a-7445e9038984 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:28:04.423602 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0388da46-37c0-4832-839a-7445e9038984 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:28:04.527095 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-0388da46-37c0-4832-839a-7445e9038984 None None] MFA Rules not processed for user `1d4f2e4dbd41441d84f03ddb30191e7a`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:28:04.528655 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-0388da46-37c0-4832-839a-7445e9038984 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:28:04.705069 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 667/1351] 10.222.0.30 () {58 vars in 1018 bytes} [Tue Sep 6 04:28:04 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 283 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 04:28:08.883262 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=59 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:28:08.912997 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782ef31c-aa42-485e-9670-99860c3725ed None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:28:08.913279 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782ef31c-aa42-485e-9670-99860c3725ed None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:28:08.913585 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782ef31c-aa42-485e-9670-99860c3725ed None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:28:08.914925 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 685/1352] 10.222.0.30 () {54 vars in 921 bytes} [Tue Sep 6 04:28:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 04:28:08.920060 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d5e17b-242a-450e-a772-68a984fabe10 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:28:08.920554 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d5e17b-242a-450e-a772-68a984fabe10 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:28:08.920675 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d5e17b-242a-450e-a772-68a984fabe10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:28:08.975965 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:28:09.000093 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-d6d5e17b-242a-450e-a772-68a984fabe10 None None] MFA Rules not processed for user `1d4f2e4dbd41441d84f03ddb30191e7a`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:28:09.001318 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-d6d5e17b-242a-450e-a772-68a984fabe10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:28:09.120641 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 668/1353] 10.222.0.30 () {58 vars in 1018 bytes} [Tue Sep 6 04:28:08 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 04:28:10.513973 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9b31bb-3798-430a-8c46-08b3f5de1017 None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:28:10.514534 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9b31bb-3798-430a-8c46-08b3f5de1017 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:28:10.514534 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9b31bb-3798-430a-8c46-08b3f5de1017 None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:28:10.515853 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 686/1354] 10.222.0.30 () {54 vars in 921 bytes} [Tue Sep 6 04:28:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 04:28:10.521772 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8deecf09-d3cc-4998-8eba-563f73d31f5f None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:28:10.522200 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8deecf09-d3cc-4998-8eba-563f73d31f5f None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:28:10.522322 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8deecf09-d3cc-4998-8eba-563f73d31f5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:28:10.598782 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-8deecf09-d3cc-4998-8eba-563f73d31f5f None None] MFA Rules not processed for user `1d4f2e4dbd41441d84f03ddb30191e7a`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:28:10.599991 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-8deecf09-d3cc-4998-8eba-563f73d31f5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:28:10.642836 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:28:10.708269 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 669/1355] 10.222.0.30 () {58 vars in 1018 bytes} [Tue Sep 6 04:28:10 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 04:28:14.171431 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb19aa9-7d21-4395-aa14-564237414ee0 None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:28:14.171927 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb19aa9-7d21-4395-aa14-564237414ee0 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:28:14.171927 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb19aa9-7d21-4395-aa14-564237414ee0 None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:28:14.173243 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 687/1356] 10.222.0.30 () {54 vars in 927 bytes} [Tue Sep 6 04:28:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 04:28:14.178597 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b84ea2c-c9a0-4216-b7c4-3bc759b600d7 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:28:14.179035 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b84ea2c-c9a0-4216-b7c4-3bc759b600d7 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:28:14.179285 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b84ea2c-c9a0-4216-b7c4-3bc759b600d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:28:14.264853 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-4b84ea2c-c9a0-4216-b7c4-3bc759b600d7 None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:28:14.266119 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-4b84ea2c-c9a0-4216-b7c4-3bc759b600d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:28:14.394266 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 670/1357] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Sep 6 04:28:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 04:28:14.713569 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27dc926c-dfd8-4de8-9ebc-c6b7c9f051a8 None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:28:14.714060 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27dc926c-dfd8-4de8-9ebc-c6b7c9f051a8 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:28:14.714509 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27dc926c-dfd8-4de8-9ebc-c6b7c9f051a8 None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:28:14.716293 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 688/1358] 10.222.0.30 () {54 vars in 962 bytes} [Tue Sep 6 04:28:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 04:28:14.726140 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14d2534-b559-48c4-9b18-f089d1fce2ad None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:28:14.726424 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14d2534-b559-48c4-9b18-f089d1fce2ad None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:28:14.726731 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14d2534-b559-48c4-9b18-f089d1fce2ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:28:14.748767 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:28:14.818279 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-a14d2534-b559-48c4-9b18-f089d1fce2ad None None] MFA Rules not processed for user `9db138b77afa4b068244b8ef715404fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:28:14.819636 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-a14d2534-b559-48c4-9b18-f089d1fce2ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:28:14.945813 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 671/1359] 10.222.0.30 () {58 vars in 1059 bytes} [Tue Sep 6 04:28:14 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 04:28:14.953614 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27dc926c-dfd8-4de8-9ebc-c6b7c9f051a8 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:28:14.955127 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-27dc926c-dfd8-4de8-9ebc-c6b7c9f051a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:28:15.106893 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27dc926c-dfd8-4de8-9ebc-c6b7c9f051a8 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:28:15.196574 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:28:15.240336 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d800b47-73be-4940-953e-d6e03dca3891 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9db138b77afa4b068244b8ef715404fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b707733ce65c42edbc1fd95ad0a1c06c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:28:15.241372 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d800b47-73be-4940-953e-d6e03dca3891 service nova] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:28:15.241927 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d800b47-73be-4940-953e-d6e03dca3891 service nova] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:28:15.242207 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d800b47-73be-4940-953e-d6e03dca3891 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:28:15.243819 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-5d800b47-73be-4940-953e-d6e03dca3891 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:28:15.339959 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d800b47-73be-4940-953e-d6e03dca3891 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:28:15.341887 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d800b47-73be-4940-953e-d6e03dca3891 service nova] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:28:15.360816 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 689/1360] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Sep 6 04:28:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 407 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 04:28:17.536596 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477894a2-a009-48f9-b674-af6ffaac5b89 None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:28:17.537031 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477894a2-a009-48f9-b674-af6ffaac5b89 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:28:17.537132 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477894a2-a009-48f9-b674-af6ffaac5b89 None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:28:17.539019 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 672/1361] 10.222.0.30 () {54 vars in 927 bytes} [Tue Sep 6 04:28:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 06 04:28:17.543904 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06353848-04aa-4b6a-b1b5-24beb74fd94b None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:28:17.544306 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06353848-04aa-4b6a-b1b5-24beb74fd94b None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:28:17.544445 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06353848-04aa-4b6a-b1b5-24beb74fd94b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:28:17.617049 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-06353848-04aa-4b6a-b1b5-24beb74fd94b None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:28:17.618315 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-06353848-04aa-4b6a-b1b5-24beb74fd94b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:28:17.660548 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:28:17.725351 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 690/1362] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Sep 6 04:28:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 04:28:18.016369 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477894a2-a009-48f9-b674-af6ffaac5b89 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:28:18.018166 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-477894a2-a009-48f9-b674-af6ffaac5b89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:28:18.087431 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:28:18.207202 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477894a2-a009-48f9-b674-af6ffaac5b89 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:28:18.336068 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14129be-d58a-42d7-bbb4-321217d0257e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9db138b77afa4b068244b8ef715404fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b707733ce65c42edbc1fd95ad0a1c06c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:28:18.337093 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14129be-d58a-42d7-bbb4-321217d0257e service nova] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:28:18.337638 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14129be-d58a-42d7-bbb4-321217d0257e service nova] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:28:18.338056 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14129be-d58a-42d7-bbb4-321217d0257e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:28:18.340198 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-c14129be-d58a-42d7-bbb4-321217d0257e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:28:18.369562 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:28:18.453606 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14129be-d58a-42d7-bbb4-321217d0257e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:28:18.455768 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14129be-d58a-42d7-bbb4-321217d0257e service nova] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:28:18.482099 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 673/1363] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Sep 6 04:28:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 466 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 04:28:20.846902 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34cf114-4996-450d-a04c-75176623943b None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:28:20.847902 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34cf114-4996-450d-a04c-75176623943b None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:28:20.848102 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34cf114-4996-450d-a04c-75176623943b None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:28:20.850068 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 691/1364] 10.222.0.30 () {54 vars in 927 bytes} [Tue Sep 6 04:28:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 04:28:20.856140 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad8169a-84cb-4a8d-86dc-57195cd2b56f None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:28:20.856583 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad8169a-84cb-4a8d-86dc-57195cd2b56f None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:28:20.856685 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad8169a-84cb-4a8d-86dc-57195cd2b56f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:28:20.930205 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-2ad8169a-84cb-4a8d-86dc-57195cd2b56f None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:28:20.931729 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-2ad8169a-84cb-4a8d-86dc-57195cd2b56f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:28:21.047237 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:28:21.053351 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 674/1365] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Sep 6 04:28:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 04:28:21.353255 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c182eed-7d79-44d3-afce-f73b58469cb8 None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:28:21.353594 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c182eed-7d79-44d3-afce-f73b58469cb8 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:28:21.353919 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c182eed-7d79-44d3-afce-f73b58469cb8 None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:28:21.355200 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 692/1366] 10.222.0.30 () {54 vars in 962 bytes} [Tue Sep 6 04:28:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 04:28:21.360942 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9bb720-5457-4c12-acb2-3f06ae7bb733 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:28:21.361291 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9bb720-5457-4c12-acb2-3f06ae7bb733 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:28:21.361883 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9bb720-5457-4c12-acb2-3f06ae7bb733 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:28:21.450837 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-2a9bb720-5457-4c12-acb2-3f06ae7bb733 None None] MFA Rules not processed for user `9db138b77afa4b068244b8ef715404fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:28:21.452044 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-2a9bb720-5457-4c12-acb2-3f06ae7bb733 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:28:21.591111 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 675/1367] 10.222.0.30 () {58 vars in 1059 bytes} [Tue Sep 6 04:28:21 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 04:28:21.599558 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c182eed-7d79-44d3-afce-f73b58469cb8 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:28:21.601242 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-8c182eed-7d79-44d3-afce-f73b58469cb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:28:21.763411 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c182eed-7d79-44d3-afce-f73b58469cb8 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:28:21.787384 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:28:21.892681 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c1e207-0228-4fd2-be2f-f16457154b27 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9db138b77afa4b068244b8ef715404fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b707733ce65c42edbc1fd95ad0a1c06c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:28:21.894891 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c1e207-0228-4fd2-be2f-f16457154b27 service nova] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:28:21.895336 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c1e207-0228-4fd2-be2f-f16457154b27 service nova] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:28:21.895841 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c1e207-0228-4fd2-be2f-f16457154b27 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:28:21.897517 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-25c1e207-0228-4fd2-be2f-f16457154b27 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:28:21.993625 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c1e207-0228-4fd2-be2f-f16457154b27 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:28:21.995498 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c1e207-0228-4fd2-be2f-f16457154b27 service nova] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:28:22.019092 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 693/1368] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Sep 6 04:28:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 420 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 04:28:22.617936 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788b506a-d932-4a98-8d81-3e4b07492c30 None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:28:22.618496 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788b506a-d932-4a98-8d81-3e4b07492c30 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:28:22.618496 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788b506a-d932-4a98-8d81-3e4b07492c30 None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:28:22.619900 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 676/1369] 10.222.0.30 () {54 vars in 927 bytes} [Tue Sep 6 04:28:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 04:28:22.625361 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9518ba45-24f5-46a8-8afd-1c8448b6ce74 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:28:22.625660 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9518ba45-24f5-46a8-8afd-1c8448b6ce74 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:28:22.625936 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9518ba45-24f5-46a8-8afd-1c8448b6ce74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:28:22.662536 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:28:22.700596 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-9518ba45-24f5-46a8-8afd-1c8448b6ce74 None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:28:22.702541 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-9518ba45-24f5-46a8-8afd-1c8448b6ce74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:28:22.809602 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 694/1370] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Sep 6 04:28:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 04:28:23.405753 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccbedb9-dda1-4f3a-a13f-b8af8ea88420 None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:28:23.406042 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccbedb9-dda1-4f3a-a13f-b8af8ea88420 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:28:23.406318 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccbedb9-dda1-4f3a-a13f-b8af8ea88420 None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:28:23.407836 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 677/1371] 10.222.0.30 () {54 vars in 964 bytes} [Tue Sep 6 04:28:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 04:28:23.412902 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f04ae6-a21c-4576-ba87-a83e8f0c2654 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:28:23.413427 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f04ae6-a21c-4576-ba87-a83e8f0c2654 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:28:23.413744 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f04ae6-a21c-4576-ba87-a83e8f0c2654 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:28:23.502791 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-84f04ae6-a21c-4576-ba87-a83e8f0c2654 None None] MFA Rules not processed for user `d09b5ab675634a2eb4b03755743f7d0d`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:28:23.504858 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-84f04ae6-a21c-4576-ba87-a83e8f0c2654 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:28:23.555711 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:28:23.589551 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 695/1372] 10.222.0.30 () {58 vars in 1061 bytes} [Tue Sep 6 04:28:23 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 04:28:23.597110 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cccbedb9-dda1-4f3a-a13f-b8af8ea88420 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:28:23.599089 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-cccbedb9-dda1-4f3a-a13f-b8af8ea88420 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:28:23.653653 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:28:23.713068 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cccbedb9-dda1-4f3a-a13f-b8af8ea88420 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:28:23.794009 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81ce1b7-3965-47e4-8c31-e27c7733e2e9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd09b5ab675634a2eb4b03755743f7d0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b707733ce65c42edbc1fd95ad0a1c06c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:28:23.794635 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81ce1b7-3965-47e4-8c31-e27c7733e2e9 service cinder] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:28:23.794918 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81ce1b7-3965-47e4-8c31-e27c7733e2e9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:28:23.795194 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81ce1b7-3965-47e4-8c31-e27c7733e2e9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:28:23.796840 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-d81ce1b7-3965-47e4-8c31-e27c7733e2e9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:28:23.918107 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:28:23.932092 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d81ce1b7-3965-47e4-8c31-e27c7733e2e9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:28:23.933154 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d81ce1b7-3965-47e4-8c31-e27c7733e2e9 service cinder] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:28:23.954161 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 678/1373] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Sep 6 04:28:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 04:28:24.246449 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c3e8e5-bffe-469e-b7a6-cc43dccb2b77 None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:28:24.246755 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c3e8e5-bffe-469e-b7a6-cc43dccb2b77 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:28:24.246990 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c3e8e5-bffe-469e-b7a6-cc43dccb2b77 None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:28:24.248342 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 696/1374] 10.222.0.30 () {54 vars in 927 bytes} [Tue Sep 6 04:28:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 04:28:24.253750 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d85172e-df6d-4ccc-8ee4-460dc3e78a11 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:28:24.254037 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d85172e-df6d-4ccc-8ee4-460dc3e78a11 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:28:24.254304 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d85172e-df6d-4ccc-8ee4-460dc3e78a11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:28:24.320739 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-8d85172e-df6d-4ccc-8ee4-460dc3e78a11 None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:28:24.321989 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-8d85172e-df6d-4ccc-8ee4-460dc3e78a11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:28:24.419481 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 679/1375] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Sep 6 04:28:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 04:28:24.713552 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c3e8e5-bffe-469e-b7a6-cc43dccb2b77 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:28:24.715150 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-67c3e8e5-bffe-469e-b7a6-cc43dccb2b77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:28:24.872464 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c3e8e5-bffe-469e-b7a6-cc43dccb2b77 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:28:24.924756 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:28:24.987148 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157d303e-4083-4c57-86e4-75b2d82bec73 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9db138b77afa4b068244b8ef715404fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b707733ce65c42edbc1fd95ad0a1c06c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:28:24.987685 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157d303e-4083-4c57-86e4-75b2d82bec73 service nova] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:28:24.988463 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157d303e-4083-4c57-86e4-75b2d82bec73 service nova] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:28:24.988717 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157d303e-4083-4c57-86e4-75b2d82bec73 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:28:24.990552 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-157d303e-4083-4c57-86e4-75b2d82bec73 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:28:25.082444 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157d303e-4083-4c57-86e4-75b2d82bec73 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:28:25.083978 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157d303e-4083-4c57-86e4-75b2d82bec73 service nova] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:28:25.102515 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 697/1376] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Sep 6 04:28:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 04:28:27.197176 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84229d99-44af-4fc3-9ab7-b7a87269f525 None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:28:27.197715 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84229d99-44af-4fc3-9ab7-b7a87269f525 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:28:27.197821 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84229d99-44af-4fc3-9ab7-b7a87269f525 None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:28:27.199096 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 680/1377] 10.222.0.30 () {54 vars in 927 bytes} [Tue Sep 6 04:28:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 04:28:27.204340 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad3a2d7-054f-437b-a4dd-de2880d83574 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:28:27.204766 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad3a2d7-054f-437b-a4dd-de2880d83574 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:28:27.204967 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad3a2d7-054f-437b-a4dd-de2880d83574 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:28:27.298303 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-4ad3a2d7-054f-437b-a4dd-de2880d83574 None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:28:27.299510 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-4ad3a2d7-054f-437b-a4dd-de2880d83574 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:28:27.306729 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:28:27.402797 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 698/1378] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Sep 6 04:28:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 04:28:27.704150 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84229d99-44af-4fc3-9ab7-b7a87269f525 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:28:27.706040 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-84229d99-44af-4fc3-9ab7-b7a87269f525 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:28:27.779456 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:28:27.877353 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84229d99-44af-4fc3-9ab7-b7a87269f525 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:28:28.006550 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48f4f69-77e5-45ff-aeae-65948a08e7d5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9db138b77afa4b068244b8ef715404fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b707733ce65c42edbc1fd95ad0a1c06c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:28:28.007166 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48f4f69-77e5-45ff-aeae-65948a08e7d5 service nova] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:28:28.007446 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48f4f69-77e5-45ff-aeae-65948a08e7d5 service nova] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:28:28.007721 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48f4f69-77e5-45ff-aeae-65948a08e7d5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:28:28.009479 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-c48f4f69-77e5-45ff-aeae-65948a08e7d5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:28:28.030820 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:28:28.116363 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48f4f69-77e5-45ff-aeae-65948a08e7d5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:28:28.118593 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48f4f69-77e5-45ff-aeae-65948a08e7d5 service nova] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:28:28.139575 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 681/1379] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Sep 6 04:28:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 436 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 04:28:30.234634 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babc1079-1612-4ceb-96af-555efc7dae4d None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:28:30.235134 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babc1079-1612-4ceb-96af-555efc7dae4d None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:28:30.235134 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babc1079-1612-4ceb-96af-555efc7dae4d None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:28:30.236592 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 699/1380] 10.222.0.30 () {54 vars in 927 bytes} [Tue Sep 6 04:28:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 04:28:30.242222 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0673c660-faf6-4c85-ab25-504573823efa None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:28:30.242638 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0673c660-faf6-4c85-ab25-504573823efa None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:28:30.242812 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0673c660-faf6-4c85-ab25-504573823efa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:28:30.313791 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-0673c660-faf6-4c85-ab25-504573823efa None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:28:30.323433 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-0673c660-faf6-4c85-ab25-504573823efa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:28:30.425217 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:28:30.431809 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 682/1381] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Sep 6 04:28:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 04:28:30.736320 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-babc1079-1612-4ceb-96af-555efc7dae4d None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:28:30.738484 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-babc1079-1612-4ceb-96af-555efc7dae4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:28:30.892035 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:28:30.902890 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-babc1079-1612-4ceb-96af-555efc7dae4d None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:28:31.024432 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e8ac1e-cfad-4617-95cd-81177ba46dda service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9db138b77afa4b068244b8ef715404fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b707733ce65c42edbc1fd95ad0a1c06c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:28:31.025121 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e8ac1e-cfad-4617-95cd-81177ba46dda service nova] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:28:31.025496 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e8ac1e-cfad-4617-95cd-81177ba46dda service nova] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:28:31.025835 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e8ac1e-cfad-4617-95cd-81177ba46dda service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:28:31.027547 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-68e8ac1e-cfad-4617-95cd-81177ba46dda service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:28:31.122952 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e8ac1e-cfad-4617-95cd-81177ba46dda service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:28:31.124655 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e8ac1e-cfad-4617-95cd-81177ba46dda service nova] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:28:31.136156 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:28:31.145704 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 700/1382] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Sep 6 04:28:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 410 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 04:28:33.475797 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aed1b7e-79a2-4539-9744-05e6fe259764 None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:28:33.476346 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aed1b7e-79a2-4539-9744-05e6fe259764 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:28:33.476503 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aed1b7e-79a2-4539-9744-05e6fe259764 None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:28:33.478338 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 683/1383] 10.222.0.30 () {54 vars in 927 bytes} [Tue Sep 6 04:28:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 04:28:33.483960 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aebad7e-26a9-4a3a-9351-89468202b39c None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:28:33.484398 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aebad7e-26a9-4a3a-9351-89468202b39c None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:28:33.484538 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aebad7e-26a9-4a3a-9351-89468202b39c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:28:33.559543 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-9aebad7e-26a9-4a3a-9351-89468202b39c None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:28:33.560818 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-9aebad7e-26a9-4a3a-9351-89468202b39c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:28:33.662500 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 701/1384] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Sep 6 04:28:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 04:28:33.958948 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aed1b7e-79a2-4539-9744-05e6fe259764 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:28:33.960763 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-8aed1b7e-79a2-4539-9744-05e6fe259764 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:28:34.125084 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aed1b7e-79a2-4539-9744-05e6fe259764 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:28:34.202493 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:28:34.253685 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf002e2-5192-4c87-99ab-22d55f06352d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9db138b77afa4b068244b8ef715404fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b707733ce65c42edbc1fd95ad0a1c06c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:28:34.254747 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf002e2-5192-4c87-99ab-22d55f06352d service nova] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:28:34.255373 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf002e2-5192-4c87-99ab-22d55f06352d service nova] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:28:34.255721 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf002e2-5192-4c87-99ab-22d55f06352d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:28:34.257825 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-ccf002e2-5192-4c87-99ab-22d55f06352d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:28:34.353617 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf002e2-5192-4c87-99ab-22d55f06352d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:28:34.355303 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf002e2-5192-4c87-99ab-22d55f06352d service nova] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:28:34.374383 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 684/1385] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Sep 6 04:28:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 416 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 04:28:36.366075 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b196b0-81c2-4b6d-8fe0-4d52c2cd0710 None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:28:36.366637 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b196b0-81c2-4b6d-8fe0-4d52c2cd0710 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:28:36.366637 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b196b0-81c2-4b6d-8fe0-4d52c2cd0710 None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:28:36.368228 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 702/1386] 10.222.0.30 () {54 vars in 927 bytes} [Tue Sep 6 04:28:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 06 04:28:36.373680 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3c1ac1-69a1-40ad-ac27-55525fd70f20 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:28:36.374113 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3c1ac1-69a1-40ad-ac27-55525fd70f20 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:28:36.374253 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3c1ac1-69a1-40ad-ac27-55525fd70f20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:28:36.447766 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-4b3c1ac1-69a1-40ad-ac27-55525fd70f20 None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:28:36.448966 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-4b3c1ac1-69a1-40ad-ac27-55525fd70f20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:28:36.484898 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:28:36.553291 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 685/1387] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Sep 6 04:28:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 04:28:36.859138 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b196b0-81c2-4b6d-8fe0-4d52c2cd0710 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:28:36.861111 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-c1b196b0-81c2-4b6d-8fe0-4d52c2cd0710 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:28:36.964917 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:28:37.038178 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b196b0-81c2-4b6d-8fe0-4d52c2cd0710 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:28:37.176398 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65291686-8158-46f7-aac6-fb4537ec5b23 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9db138b77afa4b068244b8ef715404fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b707733ce65c42edbc1fd95ad0a1c06c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:28:37.176990 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65291686-8158-46f7-aac6-fb4537ec5b23 service nova] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:28:37.177298 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65291686-8158-46f7-aac6-fb4537ec5b23 service nova] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:28:37.177624 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65291686-8158-46f7-aac6-fb4537ec5b23 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:28:37.179359 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-65291686-8158-46f7-aac6-fb4537ec5b23 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:28:37.231825 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:28:37.287785 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65291686-8158-46f7-aac6-fb4537ec5b23 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:28:37.289748 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65291686-8158-46f7-aac6-fb4537ec5b23 service nova] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:28:37.308063 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 703/1388] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Sep 6 04:28:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 450 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 04:28:39.426393 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e113913b-c60d-4d68-91de-4132a3a6f564 None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:28:39.429129 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e113913b-c60d-4d68-91de-4132a3a6f564 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:28:39.429129 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e113913b-c60d-4d68-91de-4132a3a6f564 None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:28:39.429129 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 686/1389] 10.222.0.30 () {54 vars in 927 bytes} [Tue Sep 6 04:28:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 04:28:39.430968 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eed99e6-df2f-43fb-bdbd-e6e710b20efc None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:28:39.431385 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eed99e6-df2f-43fb-bdbd-e6e710b20efc None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:28:39.431515 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eed99e6-df2f-43fb-bdbd-e6e710b20efc None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:28:39.433432 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 704/1390] 10.222.0.30 () {54 vars in 927 bytes} [Tue Sep 6 04:28:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 06 04:28:39.433908 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed378856-3ae2-45fb-b12d-f46adf8d8856 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:28:39.434198 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed378856-3ae2-45fb-b12d-f46adf8d8856 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:28:39.434536 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed378856-3ae2-45fb-b12d-f46adf8d8856 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:28:39.438180 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3331b548-c201-4af9-bccb-b76d72db107a None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:28:39.438639 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3331b548-c201-4af9-bccb-b76d72db107a None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:28:39.438917 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3331b548-c201-4af9-bccb-b76d72db107a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:28:39.502550 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-ed378856-3ae2-45fb-b12d-f46adf8d8856 None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:28:39.504498 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-ed378856-3ae2-45fb-b12d-f46adf8d8856 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:28:39.508576 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-3331b548-c201-4af9-bccb-b76d72db107a None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:28:39.509808 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-3331b548-c201-4af9-bccb-b76d72db107a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:28:39.604189 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 687/1391] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Sep 6 04:28:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 04:28:39.607745 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 705/1392] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Sep 6 04:28:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 04:28:39.899518 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed378856-3ae2-45fb-b12d-f46adf8d8856 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:28:39.900794 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-ed378856-3ae2-45fb-b12d-f46adf8d8856 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:28:39.910961 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b6c464-a02f-4ab0-90dd-3f2fac400662 None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:28:39.911281 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b6c464-a02f-4ab0-90dd-3f2fac400662 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:28:39.911558 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b6c464-a02f-4ab0-90dd-3f2fac400662 None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:28:39.912914 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 706/1393] 10.222.0.30 () {54 vars in 961 bytes} [Tue Sep 6 04:28:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 06 04:28:39.915191 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:28:39.918144 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9164c441-e412-45ba-b851-9e0614c7de4d None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:28:39.918425 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9164c441-e412-45ba-b851-9e0614c7de4d None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:28:39.918693 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9164c441-e412-45ba-b851-9e0614c7de4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:28:39.934910 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:28:39.998227 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-9164c441-e412-45ba-b851-9e0614c7de4d None None] MFA Rules not processed for user `ae5c31fb1d0342c0937bbc83a415113d`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:28:39.999449 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-9164c441-e412-45ba-b851-9e0614c7de4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:28:40.026274 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed378856-3ae2-45fb-b12d-f46adf8d8856 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:28:40.080033 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 707/1394] 10.222.0.30 () {58 vars in 1058 bytes} [Tue Sep 6 04:28:39 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 04:28:40.088088 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9164c441-e412-45ba-b851-9e0614c7de4d None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:28:40.096535 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9164c441-e412-45ba-b851-9e0614c7de4d None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:28:40.149852 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd138acc-2a8b-4118-8432-54dfd6ccece1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9db138b77afa4b068244b8ef715404fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b707733ce65c42edbc1fd95ad0a1c06c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:28:40.150495 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd138acc-2a8b-4118-8432-54dfd6ccece1 service nova] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:28:40.150773 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd138acc-2a8b-4118-8432-54dfd6ccece1 service nova] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:28:40.151081 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd138acc-2a8b-4118-8432-54dfd6ccece1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:28:40.152720 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-fd138acc-2a8b-4118-8432-54dfd6ccece1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:28:40.162690 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:28:40.174294 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b361016f-b105-422f-990e-aa1fb0415f2a service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae5c31fb1d0342c0937bbc83a415113d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b707733ce65c42edbc1fd95ad0a1c06c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:28:40.176581 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b361016f-b105-422f-990e-aa1fb0415f2a service glance] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:28:40.176581 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b361016f-b105-422f-990e-aa1fb0415f2a service glance] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:28:40.176581 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b361016f-b105-422f-990e-aa1fb0415f2a service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:28:40.177873 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-b361016f-b105-422f-990e-aa1fb0415f2a service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:28:40.212568 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:28:40.249575 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd138acc-2a8b-4118-8432-54dfd6ccece1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:28:40.251183 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd138acc-2a8b-4118-8432-54dfd6ccece1 service nova] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:28:40.272297 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 688/1395] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Sep 6 04:28:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 04:28:40.295712 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b361016f-b105-422f-990e-aa1fb0415f2a service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:28:40.297794 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b361016f-b105-422f-990e-aa1fb0415f2a service glance] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:28:40.318078 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 708/1396] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Sep 6 04:28:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 230 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 06 04:28:41.242971 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427ff9ea-ce11-493d-8d3e-dcc1ccf29811 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:28:41.243475 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427ff9ea-ce11-493d-8d3e-dcc1ccf29811 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:28:41.243580 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427ff9ea-ce11-493d-8d3e-dcc1ccf29811 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:28:41.317113 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-427ff9ea-ce11-493d-8d3e-dcc1ccf29811 None None] MFA Rules not processed for user `da5fed17101a4935bcc38b72caf9d263`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:28:41.318417 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-427ff9ea-ce11-493d-8d3e-dcc1ccf29811 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:28:41.406509 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:28:41.413330 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 689/1397] 10.222.0.30 () {58 vars in 1009 bytes} [Tue Sep 6 04:28:41 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 04:28:41.456957 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e7c74e-f2ff-455b-9d10-10b90481f4ce None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:28:41.457495 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e7c74e-f2ff-455b-9d10-10b90481f4ce None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:28:41.457763 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e7c74e-f2ff-455b-9d10-10b90481f4ce None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:28:41.459236 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 709/1398] 10.222.0.30 () {54 vars in 943 bytes} [Tue Sep 6 04:28:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 04:28:41.465890 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157ca3b8-6e06-4698-834c-314a3da90b20 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:28:41.466183 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157ca3b8-6e06-4698-834c-314a3da90b20 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:28:41.466458 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157ca3b8-6e06-4698-834c-314a3da90b20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:28:41.553076 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-157ca3b8-6e06-4698-834c-314a3da90b20 None None] MFA Rules not processed for user `a676ec7f92a144edbeca8e6029ba9ee6`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:28:41.554333 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-157ca3b8-6e06-4698-834c-314a3da90b20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:28:41.660825 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 690/1399] 10.222.0.30 () {58 vars in 1040 bytes} [Tue Sep 6 04:28:41 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 04:28:41.670121 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e7c74e-f2ff-455b-9d10-10b90481f4ce None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:28:41.671646 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-e8e7c74e-f2ff-455b-9d10-10b90481f4ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:28:41.809275 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:28:41.812515 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e7c74e-f2ff-455b-9d10-10b90481f4ce None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:28:41.920885 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fd8009-9dbf-4d52-b1e8-36c70445e786 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a676ec7f92a144edbeca8e6029ba9ee6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b707733ce65c42edbc1fd95ad0a1c06c', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:28:41.921482 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fd8009-9dbf-4d52-b1e8-36c70445e786 service swift] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:28:41.921762 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fd8009-9dbf-4d52-b1e8-36c70445e786 service swift] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:28:41.922030 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fd8009-9dbf-4d52-b1e8-36c70445e786 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:28:41.923661 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-b4fd8009-9dbf-4d52-b1e8-36c70445e786 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:28:42.011145 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4fd8009-9dbf-4d52-b1e8-36c70445e786 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:28:42.012752 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4fd8009-9dbf-4d52-b1e8-36c70445e786 service swift] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:28:42.021786 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 710/1400] 10.222.0.30 () {60 vars in 1384 bytes} [Tue Sep 6 04:28:41 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 06 04:28:42.336918 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8be5cd-17d5-49db-b920-64cfa8e40d98 None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:28:42.337461 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8be5cd-17d5-49db-b920-64cfa8e40d98 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:28:42.337575 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8be5cd-17d5-49db-b920-64cfa8e40d98 None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:28:42.338859 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 691/1401] 10.222.0.30 () {54 vars in 927 bytes} [Tue Sep 6 04:28:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 04:28:42.344273 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3a39e9-6d05-42ef-8008-13610d3b85ea None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:28:42.344554 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3a39e9-6d05-42ef-8008-13610d3b85ea None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:28:42.344823 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3a39e9-6d05-42ef-8008-13610d3b85ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:28:42.386030 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:28:42.418806 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-3b3a39e9-6d05-42ef-8008-13610d3b85ea None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:28:42.420169 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-3b3a39e9-6d05-42ef-8008-13610d3b85ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:28:42.529500 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 711/1402] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Sep 6 04:28:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 04:28:42.826037 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8be5cd-17d5-49db-b920-64cfa8e40d98 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:28:42.827683 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-5e8be5cd-17d5-49db-b920-64cfa8e40d98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:28:42.900427 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:28:42.994012 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8be5cd-17d5-49db-b920-64cfa8e40d98 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:28:43.122872 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5374a2e-29da-4492-8e0a-49c6f49ecfeb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9db138b77afa4b068244b8ef715404fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b707733ce65c42edbc1fd95ad0a1c06c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:28:43.123881 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5374a2e-29da-4492-8e0a-49c6f49ecfeb service nova] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:28:43.124423 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5374a2e-29da-4492-8e0a-49c6f49ecfeb service nova] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:28:43.124917 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5374a2e-29da-4492-8e0a-49c6f49ecfeb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:28:43.126809 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-d5374a2e-29da-4492-8e0a-49c6f49ecfeb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:28:43.147610 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:28:43.237415 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5374a2e-29da-4492-8e0a-49c6f49ecfeb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:28:43.239570 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5374a2e-29da-4492-8e0a-49c6f49ecfeb service nova] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:28:43.266165 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 692/1403] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Sep 6 04:28:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 441 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 04:28:45.489702 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58c010d-6b72-4092-8aad-79cc1ea7eca5 None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:28:45.490336 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58c010d-6b72-4092-8aad-79cc1ea7eca5 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:28:45.490336 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58c010d-6b72-4092-8aad-79cc1ea7eca5 None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:28:45.491651 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 712/1404] 10.222.0.30 () {54 vars in 927 bytes} [Tue Sep 6 04:28:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 06 04:28:45.497127 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d06d1cd-e580-4af7-8880-9ed5cdd3e8b5 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:28:45.497589 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d06d1cd-e580-4af7-8880-9ed5cdd3e8b5 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:28:45.497763 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d06d1cd-e580-4af7-8880-9ed5cdd3e8b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:28:45.572395 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-1d06d1cd-e580-4af7-8880-9ed5cdd3e8b5 None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:28:45.573664 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-1d06d1cd-e580-4af7-8880-9ed5cdd3e8b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:28:45.675792 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:28:45.683987 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 693/1405] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Sep 6 04:28:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 04:28:46.019167 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58c010d-6b72-4092-8aad-79cc1ea7eca5 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:28:46.021045 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-e58c010d-6b72-4092-8aad-79cc1ea7eca5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:28:46.158265 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:28:46.232773 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58c010d-6b72-4092-8aad-79cc1ea7eca5 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:28:46.419068 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd58a1fc-c813-419d-b5cb-145295a16b14 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9db138b77afa4b068244b8ef715404fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b707733ce65c42edbc1fd95ad0a1c06c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:28:46.420570 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd58a1fc-c813-419d-b5cb-145295a16b14 service nova] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:28:46.420570 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd58a1fc-c813-419d-b5cb-145295a16b14 service nova] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:28:46.420570 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd58a1fc-c813-419d-b5cb-145295a16b14 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:28:46.422070 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-dd58a1fc-c813-419d-b5cb-145295a16b14 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:28:46.524169 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:28:46.546814 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd58a1fc-c813-419d-b5cb-145295a16b14 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:28:46.548458 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd58a1fc-c813-419d-b5cb-145295a16b14 service nova] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:28:46.574461 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 713/1406] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Sep 6 04:28:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 556 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 04:28:48.720416 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1a8e46-26ac-429d-9e64-0a03143c9c74 None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:28:48.720952 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1a8e46-26ac-429d-9e64-0a03143c9c74 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:28:48.720952 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1a8e46-26ac-429d-9e64-0a03143c9c74 None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:28:48.723025 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 694/1407] 10.222.0.30 () {54 vars in 927 bytes} [Tue Sep 6 04:28:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 04:28:48.728170 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c63de0-d9da-4e09-a396-c31ab05d01aa None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:28:48.728586 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c63de0-d9da-4e09-a396-c31ab05d01aa None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:28:48.728718 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c63de0-d9da-4e09-a396-c31ab05d01aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:28:48.799455 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-13c63de0-d9da-4e09-a396-c31ab05d01aa None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:28:48.801221 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-13c63de0-d9da-4e09-a396-c31ab05d01aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:28:48.917061 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 714/1408] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Sep 6 04:28:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 04:28:49.205578 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1a8e46-26ac-429d-9e64-0a03143c9c74 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:28:49.207265 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-6b1a8e46-26ac-429d-9e64-0a03143c9c74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:28:49.381289 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1a8e46-26ac-429d-9e64-0a03143c9c74 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:28:49.489244 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:28:49.556188 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a082e032-3f5a-4d1c-9bdc-8e53eb75ee28 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9db138b77afa4b068244b8ef715404fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b707733ce65c42edbc1fd95ad0a1c06c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:28:49.556928 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a082e032-3f5a-4d1c-9bdc-8e53eb75ee28 service nova] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:28:49.557271 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a082e032-3f5a-4d1c-9bdc-8e53eb75ee28 service nova] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:28:49.557640 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a082e032-3f5a-4d1c-9bdc-8e53eb75ee28 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:28:49.559246 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-a082e032-3f5a-4d1c-9bdc-8e53eb75ee28 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:28:49.664139 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a082e032-3f5a-4d1c-9bdc-8e53eb75ee28 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:28:49.664139 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a082e032-3f5a-4d1c-9bdc-8e53eb75ee28 service nova] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:28:49.688049 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 695/1409] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Sep 6 04:28:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 483 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 04:28:58.914959 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=79 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:28:59.684902 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=64 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:29:06.356293 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd4e1c3-c85c-4165-9724-fc7966a9a140 None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:29:06.356870 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd4e1c3-c85c-4165-9724-fc7966a9a140 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:29:06.356870 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd4e1c3-c85c-4165-9724-fc7966a9a140 None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:29:06.358214 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 715/1410] 10.222.0.30 () {54 vars in 918 bytes} [Tue Sep 6 04:29:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 04:29:06.363934 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0009f49-c2d8-413a-8e74-832a72ce8f46 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:29:06.364456 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0009f49-c2d8-413a-8e74-832a72ce8f46 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:29:06.364943 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0009f49-c2d8-413a-8e74-832a72ce8f46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:29:06.445980 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-f0009f49-c2d8-413a-8e74-832a72ce8f46 None None] MFA Rules not processed for user `1d4f2e4dbd41441d84f03ddb30191e7a`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:29:06.447189 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-f0009f49-c2d8-413a-8e74-832a72ce8f46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:29:06.552076 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 696/1411] 10.222.0.30 () {58 vars in 1015 bytes} [Tue Sep 6 04:29:06 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 04:29:06.566089 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967543f0-5675-4ec3-bb14-ffba192c7325 None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:29:06.566373 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967543f0-5675-4ec3-bb14-ffba192c7325 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:29:06.566668 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967543f0-5675-4ec3-bb14-ffba192c7325 None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:29:06.568541 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 716/1412] 10.222.0.30 () {54 vars in 918 bytes} [Tue Sep 6 04:29:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 04:29:06.573345 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aecb482-4b26-4541-914f-6081fcba62be None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:29:06.573825 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aecb482-4b26-4541-914f-6081fcba62be None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:29:06.574106 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aecb482-4b26-4541-914f-6081fcba62be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:29:06.646883 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:29:06.649880 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-6aecb482-4b26-4541-914f-6081fcba62be None None] MFA Rules not processed for user `9db138b77afa4b068244b8ef715404fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:29:06.651136 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-6aecb482-4b26-4541-914f-6081fcba62be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:29:06.766326 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 697/1413] 10.222.0.30 () {58 vars in 1015 bytes} [Tue Sep 6 04:29:06 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 04:29:09.147746 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9230193-c822-4087-a08d-670ce8ce1ac3 None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:29:09.148316 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9230193-c822-4087-a08d-670ce8ce1ac3 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:29:09.148316 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9230193-c822-4087-a08d-670ce8ce1ac3 None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:29:09.149575 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 717/1414] 127.0.0.1 () {54 vars in 917 bytes} [Tue Sep 6 04:29:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 04:29:09.154882 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5aacc64-a985-41a4-95b2-71d73b13608d None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:29:09.155303 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5aacc64-a985-41a4-95b2-71d73b13608d None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:29:09.155463 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5aacc64-a985-41a4-95b2-71d73b13608d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:29:09.220777 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-e5aacc64-a985-41a4-95b2-71d73b13608d None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:29:09.222035 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-e5aacc64-a985-41a4-95b2-71d73b13608d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:29:09.277900 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:29:09.320525 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 698/1415] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Sep 6 04:29:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 04:29:09.616326 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9230193-c822-4087-a08d-670ce8ce1ac3 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:29:09.617918 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-d9230193-c822-4087-a08d-670ce8ce1ac3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:29:09.760970 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9230193-c822-4087-a08d-670ce8ce1ac3 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:29:09.833161 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:29:09.873034 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc50cf2f-1b32-4ac2-9e31-501606bce2aa service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9db138b77afa4b068244b8ef715404fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b707733ce65c42edbc1fd95ad0a1c06c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:29:09.873610 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc50cf2f-1b32-4ac2-9e31-501606bce2aa service nova] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:29:09.873890 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc50cf2f-1b32-4ac2-9e31-501606bce2aa service nova] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:29:09.874162 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc50cf2f-1b32-4ac2-9e31-501606bce2aa service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:29:09.875954 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-fc50cf2f-1b32-4ac2-9e31-501606bce2aa service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:29:09.971381 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc50cf2f-1b32-4ac2-9e31-501606bce2aa service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:29:09.973008 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc50cf2f-1b32-4ac2-9e31-501606bce2aa service nova] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:29:09.991566 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 718/1416] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Sep 6 04:29:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 04:29:10.890822 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b359ad-6afa-4737-a1a7-639340da0d43 None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:29:10.891253 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b359ad-6afa-4737-a1a7-639340da0d43 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:29:10.891374 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b359ad-6afa-4737-a1a7-639340da0d43 None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:29:10.892795 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 699/1417] 127.0.0.1 () {54 vars in 901 bytes} [Tue Sep 6 04:29:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 04:29:10.898122 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ca3352-fe59-492a-b677-54a5f6b8a9f5 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:29:10.898545 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ca3352-fe59-492a-b677-54a5f6b8a9f5 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:29:10.898668 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ca3352-fe59-492a-b677-54a5f6b8a9f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:29:10.972874 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-a6ca3352-fe59-492a-b677-54a5f6b8a9f5 None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:29:10.974690 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-a6ca3352-fe59-492a-b677-54a5f6b8a9f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:29:11.000982 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:29:11.069692 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 719/1418] 127.0.0.1 () {58 vars in 998 bytes} [Tue Sep 6 04:29:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 04:29:11.218313 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b359ad-6afa-4737-a1a7-639340da0d43 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:29:11.219910 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-a9b359ad-6afa-4737-a1a7-639340da0d43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:29:11.363024 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b359ad-6afa-4737-a1a7-639340da0d43 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:29:11.407100 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:29:11.484191 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2715adf-eace-4a8a-8320-fdb4f3484d4c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9db138b77afa4b068244b8ef715404fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b707733ce65c42edbc1fd95ad0a1c06c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:29:11.484722 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2715adf-eace-4a8a-8320-fdb4f3484d4c service nova] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:29:11.484999 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2715adf-eace-4a8a-8320-fdb4f3484d4c service nova] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:29:11.485265 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2715adf-eace-4a8a-8320-fdb4f3484d4c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:29:11.486879 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-e2715adf-eace-4a8a-8320-fdb4f3484d4c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:29:11.574337 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2715adf-eace-4a8a-8320-fdb4f3484d4c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:29:11.575967 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2715adf-eace-4a8a-8320-fdb4f3484d4c service nova] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:29:11.594533 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 700/1419] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Sep 6 04:29:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 04:29:12.498581 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f6ef0d-f602-4c88-9f21-d1d80c2aa1a4 None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:29:12.499143 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f6ef0d-f602-4c88-9f21-d1d80c2aa1a4 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:29:12.499143 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f6ef0d-f602-4c88-9f21-d1d80c2aa1a4 None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:29:12.500517 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 720/1420] 127.0.0.1 () {54 vars in 901 bytes} [Tue Sep 6 04:29:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 04:29:12.506031 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd4bd1f-86ea-4cc0-9b33-b1289f2252e4 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:29:12.506513 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd4bd1f-86ea-4cc0-9b33-b1289f2252e4 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:29:12.506623 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd4bd1f-86ea-4cc0-9b33-b1289f2252e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:29:12.567872 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:29:12.589916 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-0dd4bd1f-86ea-4cc0-9b33-b1289f2252e4 None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:29:12.591196 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-0dd4bd1f-86ea-4cc0-9b33-b1289f2252e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:29:12.687102 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 701/1421] 127.0.0.1 () {58 vars in 998 bytes} [Tue Sep 6 04:29:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 04:29:12.842043 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f6ef0d-f602-4c88-9f21-d1d80c2aa1a4 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:29:12.843564 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-40f6ef0d-f602-4c88-9f21-d1d80c2aa1a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:29:12.994023 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f6ef0d-f602-4c88-9f21-d1d80c2aa1a4 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:29:13.006004 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:29:13.105833 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ae8951-cf40-4c8c-836d-e8201ee11de2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9db138b77afa4b068244b8ef715404fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b707733ce65c42edbc1fd95ad0a1c06c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:29:13.106441 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ae8951-cf40-4c8c-836d-e8201ee11de2 service nova] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:29:13.106751 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ae8951-cf40-4c8c-836d-e8201ee11de2 service nova] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:29:13.107042 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ae8951-cf40-4c8c-836d-e8201ee11de2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:29:13.108634 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-26ae8951-cf40-4c8c-836d-e8201ee11de2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:29:13.195345 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ae8951-cf40-4c8c-836d-e8201ee11de2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:29:13.196945 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ae8951-cf40-4c8c-836d-e8201ee11de2 service nova] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:29:13.215252 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 721/1422] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Sep 6 04:29:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 04:29:14.079626 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0083cc6a-f393-4270-86da-e65b42fce2e6 None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:29:14.080173 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0083cc6a-f393-4270-86da-e65b42fce2e6 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:29:14.080173 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0083cc6a-f393-4270-86da-e65b42fce2e6 None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:29:14.081568 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 702/1423] 127.0.0.1 () {54 vars in 901 bytes} [Tue Sep 6 04:29:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 04:29:14.086729 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f610b1-8e49-48aa-8efa-8485f4d46e55 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:29:14.087141 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f610b1-8e49-48aa-8efa-8485f4d46e55 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:29:14.087261 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f610b1-8e49-48aa-8efa-8485f4d46e55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:29:14.107433 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:29:14.172144 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-03f610b1-8e49-48aa-8efa-8485f4d46e55 None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:29:14.173283 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-03f610b1-8e49-48aa-8efa-8485f4d46e55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:29:14.278137 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 722/1424] 127.0.0.1 () {58 vars in 998 bytes} [Tue Sep 6 04:29:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 04:29:14.440840 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0083cc6a-f393-4270-86da-e65b42fce2e6 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:29:14.440840 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-0083cc6a-f393-4270-86da-e65b42fce2e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:29:14.544757 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:29:14.581460 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0083cc6a-f393-4270-86da-e65b42fce2e6 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:29:14.691078 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd72e63b-65bb-40e8-a337-d12947b458bc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9db138b77afa4b068244b8ef715404fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b707733ce65c42edbc1fd95ad0a1c06c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:29:14.691622 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd72e63b-65bb-40e8-a337-d12947b458bc service nova] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:29:14.691902 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd72e63b-65bb-40e8-a337-d12947b458bc service nova] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:29:14.692699 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd72e63b-65bb-40e8-a337-d12947b458bc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:29:14.694701 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-cd72e63b-65bb-40e8-a337-d12947b458bc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:29:14.771845 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:29:14.785536 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd72e63b-65bb-40e8-a337-d12947b458bc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:29:14.787724 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd72e63b-65bb-40e8-a337-d12947b458bc service nova] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:29:14.806498 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 703/1425] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Sep 6 04:29:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 04:29:15.670456 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5d25d6-c736-440d-9fc4-ec3d77bb2389 None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:29:15.670986 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5d25d6-c736-440d-9fc4-ec3d77bb2389 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:29:15.670986 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5d25d6-c736-440d-9fc4-ec3d77bb2389 None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:29:15.672295 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 723/1426] 127.0.0.1 () {54 vars in 901 bytes} [Tue Sep 6 04:29:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 04:29:15.678266 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505cf525-56b8-4986-9b4e-5e0da61c6f14 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:29:15.678696 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505cf525-56b8-4986-9b4e-5e0da61c6f14 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:29:15.678793 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505cf525-56b8-4986-9b4e-5e0da61c6f14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:29:15.756287 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-505cf525-56b8-4986-9b4e-5e0da61c6f14 None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:29:15.757549 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-505cf525-56b8-4986-9b4e-5e0da61c6f14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:29:15.854786 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 704/1427] 127.0.0.1 () {58 vars in 998 bytes} [Tue Sep 6 04:29:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 04:29:16.004007 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5d25d6-c736-440d-9fc4-ec3d77bb2389 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:29:16.006517 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-2d5d25d6-c736-440d-9fc4-ec3d77bb2389 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:29:16.091139 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:29:16.156331 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5d25d6-c736-440d-9fc4-ec3d77bb2389 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:29:16.269688 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5570ede6-8922-4ca3-9346-a45a0899ce88 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9db138b77afa4b068244b8ef715404fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b707733ce65c42edbc1fd95ad0a1c06c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:29:16.270227 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5570ede6-8922-4ca3-9346-a45a0899ce88 service nova] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:29:16.270641 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5570ede6-8922-4ca3-9346-a45a0899ce88 service nova] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:29:16.271067 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5570ede6-8922-4ca3-9346-a45a0899ce88 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:29:16.272812 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-5570ede6-8922-4ca3-9346-a45a0899ce88 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:29:16.321295 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:29:16.361784 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5570ede6-8922-4ca3-9346-a45a0899ce88 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:29:16.363416 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5570ede6-8922-4ca3-9346-a45a0899ce88 service nova] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:29:16.382546 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 724/1428] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Sep 6 04:29:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 04:29:17.235803 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1d7f00-d241-442f-88ad-76bd8b22208e None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:29:17.236379 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1d7f00-d241-442f-88ad-76bd8b22208e None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:29:17.236379 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1d7f00-d241-442f-88ad-76bd8b22208e None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:29:17.237750 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 705/1429] 127.0.0.1 () {54 vars in 901 bytes} [Tue Sep 6 04:29:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 04:29:17.243880 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884af54c-27e2-4f67-81d3-220e63cdad09 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:29:17.246311 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884af54c-27e2-4f67-81d3-220e63cdad09 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:29:17.246311 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884af54c-27e2-4f67-81d3-220e63cdad09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:29:17.320886 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-884af54c-27e2-4f67-81d3-220e63cdad09 None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:29:17.322159 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-884af54c-27e2-4f67-81d3-220e63cdad09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:29:17.423318 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 725/1430] 127.0.0.1 () {58 vars in 998 bytes} [Tue Sep 6 04:29:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 04:29:17.570731 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1d7f00-d241-442f-88ad-76bd8b22208e None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:29:17.572238 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-cc1d7f00-d241-442f-88ad-76bd8b22208e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:29:17.620494 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:29:17.717458 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1d7f00-d241-442f-88ad-76bd8b22208e None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:29:17.827974 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab393b85-f827-4150-a0c8-e2b626f1672b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9db138b77afa4b068244b8ef715404fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b707733ce65c42edbc1fd95ad0a1c06c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:29:17.828577 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab393b85-f827-4150-a0c8-e2b626f1672b service nova] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:29:17.828853 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab393b85-f827-4150-a0c8-e2b626f1672b service nova] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:29:17.829121 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab393b85-f827-4150-a0c8-e2b626f1672b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:29:17.830732 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-ab393b85-f827-4150-a0c8-e2b626f1672b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:29:17.838180 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:29:17.919143 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab393b85-f827-4150-a0c8-e2b626f1672b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:29:17.920731 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab393b85-f827-4150-a0c8-e2b626f1672b service nova] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:29:17.938296 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 706/1431] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Sep 6 04:29:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 04:29:18.806013 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67da1828-8724-462e-912c-055f8929458e None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:29:18.806561 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67da1828-8724-462e-912c-055f8929458e None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:29:18.806561 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67da1828-8724-462e-912c-055f8929458e None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:29:18.807948 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 726/1432] 127.0.0.1 () {54 vars in 901 bytes} [Tue Sep 6 04:29:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 04:29:18.812817 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a81ff9a-643d-4a55-9fef-001918422b90 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:29:18.813214 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a81ff9a-643d-4a55-9fef-001918422b90 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:29:18.813395 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a81ff9a-643d-4a55-9fef-001918422b90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:29:18.891967 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-8a81ff9a-643d-4a55-9fef-001918422b90 None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:29:18.893239 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-8a81ff9a-643d-4a55-9fef-001918422b90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:29:18.963077 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:29:18.991740 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 707/1433] 127.0.0.1 () {58 vars in 998 bytes} [Tue Sep 6 04:29:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 04:29:19.139937 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67da1828-8724-462e-912c-055f8929458e None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:29:19.141421 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-67da1828-8724-462e-912c-055f8929458e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:29:19.159935 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:29:19.287971 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67da1828-8724-462e-912c-055f8929458e None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:29:19.375794 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:29:19.402364 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e5a799-797b-4430-96b4-f3fb267fe25b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9db138b77afa4b068244b8ef715404fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b707733ce65c42edbc1fd95ad0a1c06c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:29:19.402923 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e5a799-797b-4430-96b4-f3fb267fe25b service nova] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:29:19.403208 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e5a799-797b-4430-96b4-f3fb267fe25b service nova] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:29:19.403484 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e5a799-797b-4430-96b4-f3fb267fe25b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:29:19.405105 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-08e5a799-797b-4430-96b4-f3fb267fe25b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:29:19.496528 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e5a799-797b-4430-96b4-f3fb267fe25b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:29:19.498182 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e5a799-797b-4430-96b4-f3fb267fe25b service nova] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:29:19.515931 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 727/1434] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Sep 6 04:29:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 04:29:20.397840 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8925671b-7c99-49a5-98dc-ef7aeacca711 None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:29:20.398288 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8925671b-7c99-49a5-98dc-ef7aeacca711 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:29:20.398402 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8925671b-7c99-49a5-98dc-ef7aeacca711 None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:29:20.399758 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 708/1435] 127.0.0.1 () {54 vars in 901 bytes} [Tue Sep 6 04:29:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 04:29:20.404748 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97901377-cd8c-4f18-8967-a281f3486c23 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:29:20.405207 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97901377-cd8c-4f18-8967-a281f3486c23 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:29:20.405338 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97901377-cd8c-4f18-8967-a281f3486c23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:29:20.482989 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-97901377-cd8c-4f18-8967-a281f3486c23 None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:29:20.484203 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-97901377-cd8c-4f18-8967-a281f3486c23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:29:20.527787 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:29:20.585518 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 728/1436] 127.0.0.1 () {58 vars in 998 bytes} [Tue Sep 6 04:29:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 04:29:20.738371 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8925671b-7c99-49a5-98dc-ef7aeacca711 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:29:20.740463 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-8925671b-7c99-49a5-98dc-ef7aeacca711 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:29:20.885174 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8925671b-7c99-49a5-98dc-ef7aeacca711 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:29:20.939256 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:29:20.997398 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a73d09a-44d4-4da2-af08-02872e8a44d3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9db138b77afa4b068244b8ef715404fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b707733ce65c42edbc1fd95ad0a1c06c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:29:20.998011 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a73d09a-44d4-4da2-af08-02872e8a44d3 service nova] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:29:20.998291 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a73d09a-44d4-4da2-af08-02872e8a44d3 service nova] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:29:20.998562 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a73d09a-44d4-4da2-af08-02872e8a44d3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:29:21.000163 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-1a73d09a-44d4-4da2-af08-02872e8a44d3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:29:21.090299 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a73d09a-44d4-4da2-af08-02872e8a44d3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:29:21.091902 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a73d09a-44d4-4da2-af08-02872e8a44d3 service nova] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:29:21.109636 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 709/1437] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Sep 6 04:29:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 04:29:21.962256 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5a8b91-6302-40b9-9c87-39b15761a7c5 None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:29:21.962828 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5a8b91-6302-40b9-9c87-39b15761a7c5 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:29:21.962828 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5a8b91-6302-40b9-9c87-39b15761a7c5 None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:29:21.964135 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 729/1438] 127.0.0.1 () {54 vars in 901 bytes} [Tue Sep 6 04:29:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 04:29:21.968945 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0707e03-f942-4e1d-b0bd-87c3f381d9f6 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:29:21.969401 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0707e03-f942-4e1d-b0bd-87c3f381d9f6 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:29:21.969645 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0707e03-f942-4e1d-b0bd-87c3f381d9f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:29:22.047890 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-f0707e03-f942-4e1d-b0bd-87c3f381d9f6 None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:29:22.049118 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-f0707e03-f942-4e1d-b0bd-87c3f381d9f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:29:22.051153 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:29:22.147817 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 710/1439] 127.0.0.1 () {58 vars in 998 bytes} [Tue Sep 6 04:29:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 04:29:22.298266 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5a8b91-6302-40b9-9c87-39b15761a7c5 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:29:22.299697 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-bc5a8b91-6302-40b9-9c87-39b15761a7c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:29:22.445296 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5a8b91-6302-40b9-9c87-39b15761a7c5 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:29:22.471928 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:29:22.560610 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d327f59f-a6ce-4ade-930a-9374a08f5f67 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9db138b77afa4b068244b8ef715404fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b707733ce65c42edbc1fd95ad0a1c06c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:29:22.561143 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d327f59f-a6ce-4ade-930a-9374a08f5f67 service nova] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:29:22.561465 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d327f59f-a6ce-4ade-930a-9374a08f5f67 service nova] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:29:22.561745 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d327f59f-a6ce-4ade-930a-9374a08f5f67 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:29:22.563350 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-d327f59f-a6ce-4ade-930a-9374a08f5f67 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:29:22.650780 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d327f59f-a6ce-4ade-930a-9374a08f5f67 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:29:22.652392 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d327f59f-a6ce-4ade-930a-9374a08f5f67 service nova] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:29:22.670553 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 730/1440] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Sep 6 04:29:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 04:29:24.503669 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f49528-8ce0-4004-8fda-803eef7e032f None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:29:24.504218 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f49528-8ce0-4004-8fda-803eef7e032f None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:29:24.504350 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f49528-8ce0-4004-8fda-803eef7e032f None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:29:24.506090 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 711/1441] 127.0.0.1 () {54 vars in 917 bytes} [Tue Sep 6 04:29:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 04:29:24.511395 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ac7838-a8a7-4aa0-ad21-236c2db21231 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:29:24.511847 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ac7838-a8a7-4aa0-ad21-236c2db21231 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:29:24.511944 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ac7838-a8a7-4aa0-ad21-236c2db21231 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:29:24.554400 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:29:24.579109 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-67ac7838-a8a7-4aa0-ad21-236c2db21231 None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:29:24.580339 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-67ac7838-a8a7-4aa0-ad21-236c2db21231 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:29:24.677920 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 731/1442] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Sep 6 04:29:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 04:29:24.969109 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f49528-8ce0-4004-8fda-803eef7e032f None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:29:24.970683 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-62f49528-8ce0-4004-8fda-803eef7e032f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:29:25.075752 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f49528-8ce0-4004-8fda-803eef7e032f None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:29:25.108958 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:29:25.151663 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34c356c-380a-43a3-a196-466126497167 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90492440f6a6471e8392361c098428aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b707733ce65c42edbc1fd95ad0a1c06c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:29:25.152268 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34c356c-380a-43a3-a196-466126497167 service neutron] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:29:25.152544 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34c356c-380a-43a3-a196-466126497167 service neutron] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:29:25.152811 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34c356c-380a-43a3-a196-466126497167 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:29:25.154432 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-a34c356c-380a-43a3-a196-466126497167 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:29:25.271692 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34c356c-380a-43a3-a196-466126497167 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:29:25.273061 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34c356c-380a-43a3-a196-466126497167 service neutron] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:29:25.292465 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 712/1443] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Sep 6 04:29:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 04:29:27.392652 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2c1aad-bf39-473b-add4-3a9bfa638fa0 None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:29:27.393122 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2c1aad-bf39-473b-add4-3a9bfa638fa0 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:29:27.393226 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2c1aad-bf39-473b-add4-3a9bfa638fa0 None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:29:27.394867 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 732/1444] 127.0.0.1 () {54 vars in 917 bytes} [Tue Sep 6 04:29:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 04:29:27.400234 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad1ae8c-65e1-475b-9e7e-53fc5a71cd38 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:29:27.400682 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad1ae8c-65e1-475b-9e7e-53fc5a71cd38 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:29:27.400787 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad1ae8c-65e1-475b-9e7e-53fc5a71cd38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:29:27.470627 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-aad1ae8c-65e1-475b-9e7e-53fc5a71cd38 None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:29:27.471768 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-aad1ae8c-65e1-475b-9e7e-53fc5a71cd38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:29:27.492403 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:29:27.586027 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 713/1445] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Sep 6 04:29:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 04:29:27.886722 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2c1aad-bf39-473b-add4-3a9bfa638fa0 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:29:27.888270 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-fb2c1aad-bf39-473b-add4-3a9bfa638fa0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:29:27.996871 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2c1aad-bf39-473b-add4-3a9bfa638fa0 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:29:28.009671 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:29:28.070536 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84350371-b791-4990-a3fa-b6bccab1b0d9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90492440f6a6471e8392361c098428aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b707733ce65c42edbc1fd95ad0a1c06c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:29:28.071179 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84350371-b791-4990-a3fa-b6bccab1b0d9 service neutron] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:29:28.071460 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84350371-b791-4990-a3fa-b6bccab1b0d9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:29:28.071727 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84350371-b791-4990-a3fa-b6bccab1b0d9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:29:28.073387 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-84350371-b791-4990-a3fa-b6bccab1b0d9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:29:28.184849 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84350371-b791-4990-a3fa-b6bccab1b0d9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:29:28.186248 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84350371-b791-4990-a3fa-b6bccab1b0d9 service neutron] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:29:28.204919 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 733/1446] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Sep 6 04:29:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 04:29:30.502237 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6914108-82fa-4f74-a259-7ed89827351c None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:29:30.502758 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6914108-82fa-4f74-a259-7ed89827351c None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:29:30.502758 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6914108-82fa-4f74-a259-7ed89827351c None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:29:30.504128 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 714/1447] 127.0.0.1 () {54 vars in 917 bytes} [Tue Sep 6 04:29:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 04:29:30.509412 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f552b243-adec-4505-a911-2dbcf25749f6 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:29:30.509858 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f552b243-adec-4505-a911-2dbcf25749f6 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:29:30.510016 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f552b243-adec-4505-a911-2dbcf25749f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:29:30.558417 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:29:30.576366 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-f552b243-adec-4505-a911-2dbcf25749f6 None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:29:30.577595 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-f552b243-adec-4505-a911-2dbcf25749f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:29:30.675298 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 734/1448] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Sep 6 04:29:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 04:29:30.963900 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6914108-82fa-4f74-a259-7ed89827351c None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:29:30.965516 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-d6914108-82fa-4f74-a259-7ed89827351c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:29:31.069701 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6914108-82fa-4f74-a259-7ed89827351c None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:29:31.109470 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:29:31.142906 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae34e1f5-7471-4689-85e1-1c587bf06415 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90492440f6a6471e8392361c098428aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b707733ce65c42edbc1fd95ad0a1c06c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:29:31.143770 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae34e1f5-7471-4689-85e1-1c587bf06415 service neutron] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:29:31.144054 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae34e1f5-7471-4689-85e1-1c587bf06415 service neutron] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:29:31.144328 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae34e1f5-7471-4689-85e1-1c587bf06415 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:29:31.147687 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-ae34e1f5-7471-4689-85e1-1c587bf06415 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:29:31.255757 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae34e1f5-7471-4689-85e1-1c587bf06415 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:29:31.257112 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae34e1f5-7471-4689-85e1-1c587bf06415 service neutron] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:29:31.276092 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 715/1449] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Sep 6 04:29:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 04:29:33.387896 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2da52ea-75f1-415c-9766-f2ee774aaf9e None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:29:33.388352 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2da52ea-75f1-415c-9766-f2ee774aaf9e None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:29:33.388461 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2da52ea-75f1-415c-9766-f2ee774aaf9e None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:29:33.389849 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 735/1450] 127.0.0.1 () {54 vars in 917 bytes} [Tue Sep 6 04:29:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 04:29:33.395979 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed304a8b-70af-47f5-b652-bc188056fd74 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:29:33.396502 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed304a8b-70af-47f5-b652-bc188056fd74 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:29:33.396502 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed304a8b-70af-47f5-b652-bc188056fd74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:29:33.461350 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-ed304a8b-70af-47f5-b652-bc188056fd74 None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:29:33.462564 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-ed304a8b-70af-47f5-b652-bc188056fd74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:29:33.489301 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:29:33.565030 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 716/1451] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Sep 6 04:29:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 04:29:33.849895 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2da52ea-75f1-415c-9766-f2ee774aaf9e None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:29:33.851449 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-a2da52ea-75f1-415c-9766-f2ee774aaf9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:29:33.955447 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2da52ea-75f1-415c-9766-f2ee774aaf9e None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:29:33.975058 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:29:34.030094 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055a1eaa-ef56-47f9-bdba-d592d4a9f132 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90492440f6a6471e8392361c098428aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b707733ce65c42edbc1fd95ad0a1c06c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:29:34.030738 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055a1eaa-ef56-47f9-bdba-d592d4a9f132 service neutron] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:29:34.031014 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055a1eaa-ef56-47f9-bdba-d592d4a9f132 service neutron] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:29:34.031286 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055a1eaa-ef56-47f9-bdba-d592d4a9f132 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:29:34.032894 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-055a1eaa-ef56-47f9-bdba-d592d4a9f132 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:29:34.141500 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055a1eaa-ef56-47f9-bdba-d592d4a9f132 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:29:34.142867 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055a1eaa-ef56-47f9-bdba-d592d4a9f132 service neutron] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:29:34.161264 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 736/1452] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Sep 6 04:29:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 04:29:36.126865 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8616fa2c-e7d3-4ac7-bfc4-b82753fc77da None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:29:36.127431 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8616fa2c-e7d3-4ac7-bfc4-b82753fc77da None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:29:36.127431 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8616fa2c-e7d3-4ac7-bfc4-b82753fc77da None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:29:36.128726 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 717/1453] 127.0.0.1 () {54 vars in 917 bytes} [Tue Sep 6 04:29:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 04:29:36.134165 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac79e75-d659-42fd-9021-1501011aa189 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:29:36.134591 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac79e75-d659-42fd-9021-1501011aa189 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:29:36.134725 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac79e75-d659-42fd-9021-1501011aa189 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:29:36.204381 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:29:36.207105 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-9ac79e75-d659-42fd-9021-1501011aa189 None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:29:36.208988 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-9ac79e75-d659-42fd-9021-1501011aa189 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:29:36.307994 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 737/1454] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Sep 6 04:29:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 04:29:36.591709 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8616fa2c-e7d3-4ac7-bfc4-b82753fc77da None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:29:36.593329 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-8616fa2c-e7d3-4ac7-bfc4-b82753fc77da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:29:36.816864 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8616fa2c-e7d3-4ac7-bfc4-b82753fc77da None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:29:36.865736 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:29:36.890360 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426dc269-c6aa-4466-9c1c-b59d49bde720 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90492440f6a6471e8392361c098428aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b707733ce65c42edbc1fd95ad0a1c06c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:29:36.890706 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426dc269-c6aa-4466-9c1c-b59d49bde720 service neutron] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:29:36.890985 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426dc269-c6aa-4466-9c1c-b59d49bde720 service neutron] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:29:36.891251 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426dc269-c6aa-4466-9c1c-b59d49bde720 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:29:36.892849 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-426dc269-c6aa-4466-9c1c-b59d49bde720 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:29:37.002454 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-426dc269-c6aa-4466-9c1c-b59d49bde720 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:29:37.003877 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-426dc269-c6aa-4466-9c1c-b59d49bde720 service neutron] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:29:37.022799 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 718/1455] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Sep 6 04:29:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 432 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 04:29:39.019265 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2e8844-be1e-4428-a4fd-f73dbe34da9e None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:29:39.019842 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2e8844-be1e-4428-a4fd-f73dbe34da9e None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:29:39.019842 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2e8844-be1e-4428-a4fd-f73dbe34da9e None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:29:39.021599 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 738/1456] 127.0.0.1 () {54 vars in 917 bytes} [Tue Sep 6 04:29:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 04:29:39.026940 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28cbcc7-a3b9-46df-9e7f-aeb4057ed678 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:29:39.027376 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28cbcc7-a3b9-46df-9e7f-aeb4057ed678 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:29:39.027502 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28cbcc7-a3b9-46df-9e7f-aeb4057ed678 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:29:39.093752 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-f28cbcc7-a3b9-46df-9e7f-aeb4057ed678 None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:29:39.094970 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-f28cbcc7-a3b9-46df-9e7f-aeb4057ed678 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:29:39.124698 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:29:39.191689 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 719/1457] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Sep 6 04:29:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 04:29:39.477530 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2e8844-be1e-4428-a4fd-f73dbe34da9e None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:29:39.479099 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-9d2e8844-be1e-4428-a4fd-f73dbe34da9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:29:39.585075 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2e8844-be1e-4428-a4fd-f73dbe34da9e None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:29:39.616481 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:29:39.661879 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19bb9ad0-7d5c-442f-88d5-725ef3aa5428 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90492440f6a6471e8392361c098428aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b707733ce65c42edbc1fd95ad0a1c06c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:29:39.662481 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bb9ad0-7d5c-442f-88d5-725ef3aa5428 service neutron] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:29:39.662786 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bb9ad0-7d5c-442f-88d5-725ef3aa5428 service neutron] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:29:39.663055 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bb9ad0-7d5c-442f-88d5-725ef3aa5428 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:29:39.664753 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-19bb9ad0-7d5c-442f-88d5-725ef3aa5428 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:29:39.891341 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19bb9ad0-7d5c-442f-88d5-725ef3aa5428 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:29:39.892773 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19bb9ad0-7d5c-442f-88d5-725ef3aa5428 service neutron] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:29:39.912620 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 739/1458] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Sep 6 04:29:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 436 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 04:29:41.907467 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c915416a-9589-4a57-a68b-acfe60477640 None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:29:41.908033 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c915416a-9589-4a57-a68b-acfe60477640 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:29:41.908033 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c915416a-9589-4a57-a68b-acfe60477640 None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:29:41.909372 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 720/1459] 127.0.0.1 () {54 vars in 917 bytes} [Tue Sep 6 04:29:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 04:29:41.915202 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea8f0ee-a856-445a-bd7f-9bede4a3f768 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:29:41.915606 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea8f0ee-a856-445a-bd7f-9bede4a3f768 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:29:41.915775 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea8f0ee-a856-445a-bd7f-9bede4a3f768 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:29:41.980904 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-0ea8f0ee-a856-445a-bd7f-9bede4a3f768 None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:29:41.982114 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-0ea8f0ee-a856-445a-bd7f-9bede4a3f768 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:29:41.990768 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:29:42.077875 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 740/1460] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Sep 6 04:29:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 04:29:42.366912 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c915416a-9589-4a57-a68b-acfe60477640 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:29:42.368550 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-c915416a-9589-4a57-a68b-acfe60477640 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:29:42.470831 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c915416a-9589-4a57-a68b-acfe60477640 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:29:42.524613 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:29:42.543238 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1426d856-48c1-413d-87b6-7db5b5d5409a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90492440f6a6471e8392361c098428aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b707733ce65c42edbc1fd95ad0a1c06c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:29:42.543849 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1426d856-48c1-413d-87b6-7db5b5d5409a service neutron] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:29:42.544126 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1426d856-48c1-413d-87b6-7db5b5d5409a service neutron] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:29:42.544392 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1426d856-48c1-413d-87b6-7db5b5d5409a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:29:42.546020 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-1426d856-48c1-413d-87b6-7db5b5d5409a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:29:42.663371 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1426d856-48c1-413d-87b6-7db5b5d5409a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:29:42.664694 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1426d856-48c1-413d-87b6-7db5b5d5409a service neutron] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:29:42.683325 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 721/1461] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Sep 6 04:29:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 04:29:44.673329 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5927e309-513c-4e15-a5e6-cf1137126d96 None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:29:44.673833 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5927e309-513c-4e15-a5e6-cf1137126d96 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:29:44.673833 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5927e309-513c-4e15-a5e6-cf1137126d96 None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:29:44.675152 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 741/1462] 127.0.0.1 () {54 vars in 917 bytes} [Tue Sep 6 04:29:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 04:29:44.681219 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d30bdb-6ccc-45b1-9dac-70c7b9667dfa None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:29:44.681659 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d30bdb-6ccc-45b1-9dac-70c7b9667dfa None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:29:44.681791 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d30bdb-6ccc-45b1-9dac-70c7b9667dfa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:29:44.746538 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-f4d30bdb-6ccc-45b1-9dac-70c7b9667dfa None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:29:44.747761 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-f4d30bdb-6ccc-45b1-9dac-70c7b9667dfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:29:44.787825 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:29:44.846165 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 722/1463] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Sep 6 04:29:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 04:29:45.130281 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5927e309-513c-4e15-a5e6-cf1137126d96 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:29:45.132050 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-5927e309-513c-4e15-a5e6-cf1137126d96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:29:45.238465 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5927e309-513c-4e15-a5e6-cf1137126d96 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:29:45.275606 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:29:45.310222 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c5ff41-5c48-43c0-b466-ae12890a27f1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90492440f6a6471e8392361c098428aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b707733ce65c42edbc1fd95ad0a1c06c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:29:45.310821 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c5ff41-5c48-43c0-b466-ae12890a27f1 service neutron] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:29:45.311166 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c5ff41-5c48-43c0-b466-ae12890a27f1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:29:45.311437 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c5ff41-5c48-43c0-b466-ae12890a27f1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:29:45.313016 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-48c5ff41-5c48-43c0-b466-ae12890a27f1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:29:45.423282 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c5ff41-5c48-43c0-b466-ae12890a27f1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:29:45.424754 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c5ff41-5c48-43c0-b466-ae12890a27f1 service neutron] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:29:45.443573 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 742/1464] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Sep 6 04:29:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 04:29:47.496844 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e76d1ea-ad61-498b-9d4c-1a2acdbd7848 None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:29:47.497403 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e76d1ea-ad61-498b-9d4c-1a2acdbd7848 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:29:47.497403 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e76d1ea-ad61-498b-9d4c-1a2acdbd7848 None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:29:47.498742 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 723/1465] 127.0.0.1 () {54 vars in 917 bytes} [Tue Sep 6 04:29:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 04:29:47.507310 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669c18fe-16db-4ba3-a69f-72e323286824 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:29:47.507842 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669c18fe-16db-4ba3-a69f-72e323286824 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:29:47.507943 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669c18fe-16db-4ba3-a69f-72e323286824 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:29:47.595786 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-669c18fe-16db-4ba3-a69f-72e323286824 None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:29:47.598528 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-669c18fe-16db-4ba3-a69f-72e323286824 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:29:47.618545 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:29:47.712623 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 743/1466] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Sep 6 04:29:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 04:29:48.006052 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e76d1ea-ad61-498b-9d4c-1a2acdbd7848 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:29:48.007612 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-1e76d1ea-ad61-498b-9d4c-1a2acdbd7848 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:29:48.112371 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e76d1ea-ad61-498b-9d4c-1a2acdbd7848 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:29:48.179170 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:29:48.185781 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebdd0ea-5248-4967-be6e-ceeeeeb4c8fa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90492440f6a6471e8392361c098428aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b707733ce65c42edbc1fd95ad0a1c06c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:29:48.186393 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebdd0ea-5248-4967-be6e-ceeeeeb4c8fa service neutron] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:29:48.186674 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebdd0ea-5248-4967-be6e-ceeeeeb4c8fa service neutron] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:29:48.186944 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebdd0ea-5248-4967-be6e-ceeeeeb4c8fa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:29:48.188550 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-aebdd0ea-5248-4967-be6e-ceeeeeb4c8fa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:29:48.300234 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aebdd0ea-5248-4967-be6e-ceeeeeb4c8fa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:29:48.301771 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aebdd0ea-5248-4967-be6e-ceeeeeb4c8fa service neutron] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:29:48.319283 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 724/1467] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Sep 6 04:29:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 04:29:50.234664 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b452781a-8da5-4dfe-9327-bf8415f5f78a None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:29:50.235223 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b452781a-8da5-4dfe-9327-bf8415f5f78a None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:29:50.235223 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b452781a-8da5-4dfe-9327-bf8415f5f78a None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:29:50.236558 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 744/1468] 127.0.0.1 () {54 vars in 917 bytes} [Tue Sep 6 04:29:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 04:29:50.242748 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae88736-884a-4fc1-8224-e96125b3a946 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:29:50.243155 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae88736-884a-4fc1-8224-e96125b3a946 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:29:50.243296 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae88736-884a-4fc1-8224-e96125b3a946 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:29:50.311428 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-fae88736-884a-4fc1-8224-e96125b3a946 None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:29:50.312987 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-fae88736-884a-4fc1-8224-e96125b3a946 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:29:50.361368 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:29:50.411277 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 725/1469] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Sep 6 04:29:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 04:29:50.699824 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b452781a-8da5-4dfe-9327-bf8415f5f78a None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:29:50.701460 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-b452781a-8da5-4dfe-9327-bf8415f5f78a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:29:50.802932 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b452781a-8da5-4dfe-9327-bf8415f5f78a None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:29:50.849255 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:29:50.874888 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14900443-20c5-4c42-8f84-3372dd8e8d18 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90492440f6a6471e8392361c098428aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b707733ce65c42edbc1fd95ad0a1c06c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:29:50.875486 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14900443-20c5-4c42-8f84-3372dd8e8d18 service neutron] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:29:50.875762 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14900443-20c5-4c42-8f84-3372dd8e8d18 service neutron] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:29:50.876029 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14900443-20c5-4c42-8f84-3372dd8e8d18 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:29:50.877724 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-14900443-20c5-4c42-8f84-3372dd8e8d18 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:29:50.984373 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14900443-20c5-4c42-8f84-3372dd8e8d18 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:29:50.985774 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14900443-20c5-4c42-8f84-3372dd8e8d18 service neutron] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:29:51.005154 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 745/1470] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Sep 6 04:29:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 04:30:00.449121 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=22 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:30:01.002898 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=66 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:30:10.752374 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a706cdbf-50b9-4f6a-b7b8-66a786d1e451 None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:30:10.752903 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a706cdbf-50b9-4f6a-b7b8-66a786d1e451 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:30:10.752903 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a706cdbf-50b9-4f6a-b7b8-66a786d1e451 None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:30:10.754820 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 726/1471] 10.222.0.30 () {54 vars in 927 bytes} [Tue Sep 6 04:30:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 04:30:10.760235 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c10281-0c6c-455c-997c-bdef3a3c4979 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:30:10.760681 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c10281-0c6c-455c-997c-bdef3a3c4979 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:30:10.760794 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c10281-0c6c-455c-997c-bdef3a3c4979 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:30:10.826150 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-24c10281-0c6c-455c-997c-bdef3a3c4979 None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:30:10.827336 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-24c10281-0c6c-455c-997c-bdef3a3c4979 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:30:10.928772 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 746/1472] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Sep 6 04:30:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 04:30:11.232223 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65538df-569a-481d-a53a-63ace174b753 None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:30:11.232534 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65538df-569a-481d-a53a-63ace174b753 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:30:11.232813 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65538df-569a-481d-a53a-63ace174b753 None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:30:11.234228 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 727/1473] 10.222.0.30 () {54 vars in 961 bytes} [Tue Sep 6 04:30:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 06 04:30:11.239971 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11df9a2b-3d65-4b54-bbcd-10fc62f927f4 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:30:11.240256 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11df9a2b-3d65-4b54-bbcd-10fc62f927f4 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:30:11.240554 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11df9a2b-3d65-4b54-bbcd-10fc62f927f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:30:11.315370 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-11df9a2b-3d65-4b54-bbcd-10fc62f927f4 None None] MFA Rules not processed for user `ae5c31fb1d0342c0937bbc83a415113d`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:30:11.317170 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-11df9a2b-3d65-4b54-bbcd-10fc62f927f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:30:11.336778 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:30:11.392333 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 747/1474] 10.222.0.30 () {58 vars in 1058 bytes} [Tue Sep 6 04:30:11 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 04:30:11.400706 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65538df-569a-481d-a53a-63ace174b753 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:30:11.402246 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-d65538df-569a-481d-a53a-63ace174b753 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:30:11.505293 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65538df-569a-481d-a53a-63ace174b753 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:30:11.589155 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f63dd6e-7c74-43ca-b15c-56b1b76e5ab4 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae5c31fb1d0342c0937bbc83a415113d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b707733ce65c42edbc1fd95ad0a1c06c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:30:11.589809 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f63dd6e-7c74-43ca-b15c-56b1b76e5ab4 service glance] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:30:11.590090 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f63dd6e-7c74-43ca-b15c-56b1b76e5ab4 service glance] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:30:11.590459 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f63dd6e-7c74-43ca-b15c-56b1b76e5ab4 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:30:11.592085 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-1f63dd6e-7c74-43ca-b15c-56b1b76e5ab4 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:30:11.659140 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:30:11.711391 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f63dd6e-7c74-43ca-b15c-56b1b76e5ab4 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:30:11.712837 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f63dd6e-7c74-43ca-b15c-56b1b76e5ab4 service glance] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:30:11.731101 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 728/1475] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Sep 6 04:30:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 04:30:13.795038 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe6cc87-8b4f-4282-b91c-1d4345e508c1 None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:30:13.795557 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe6cc87-8b4f-4282-b91c-1d4345e508c1 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:30:13.795557 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe6cc87-8b4f-4282-b91c-1d4345e508c1 None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:30:13.796895 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 748/1476] 10.222.0.30 () {54 vars in 927 bytes} [Tue Sep 6 04:30:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 04:30:13.802528 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0eedf85-6016-4756-9716-d06fb5849eb5 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:30:13.802978 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0eedf85-6016-4756-9716-d06fb5849eb5 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:30:13.803076 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0eedf85-6016-4756-9716-d06fb5849eb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:30:13.870957 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-f0eedf85-6016-4756-9716-d06fb5849eb5 None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:30:13.872211 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-f0eedf85-6016-4756-9716-d06fb5849eb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:30:13.972375 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 729/1477] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Sep 6 04:30:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 04:30:14.262697 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe6cc87-8b4f-4282-b91c-1d4345e508c1 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:30:14.264234 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-dbe6cc87-8b4f-4282-b91c-1d4345e508c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:30:14.399350 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe6cc87-8b4f-4282-b91c-1d4345e508c1 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:30:14.450061 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:30:14.496305 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8935a8e-c91b-4bc5-8ac7-9a83e26555ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:30:14.496920 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8935a8e-c91b-4bc5-8ac7-9a83e26555ef admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:30:14.497195 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8935a8e-c91b-4bc5-8ac7-9a83e26555ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:30:14.497540 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8935a8e-c91b-4bc5-8ac7-9a83e26555ef admin admin] PATH_INFO: `/v3/projects` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:30:14.498209 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8935a8e-c91b-4bc5-8ac7-9a83e26555ef admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:30:14.499949 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: WARNING py.warnings [None req-e8935a8e-c91b-4bc5-8ac7-9a83e26555ef admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 06 04:30:14.499949 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: warnings.warn(msg) Sep 06 04:30:14.499949 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: Sep 06 04:30:14.501088 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8935a8e-c91b-4bc5-8ac7-9a83e26555ef admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:30:14.521749 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 749/1478] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Sep 6 04:30:14 2022] GET /identity/v3/projects => generated 2553 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 04:30:16.466165 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7d7e9a-183b-4673-bff2-e29b96034abb None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:30:16.466697 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7d7e9a-183b-4673-bff2-e29b96034abb None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:30:16.466697 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7d7e9a-183b-4673-bff2-e29b96034abb None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:30:16.468058 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 730/1479] 10.222.0.30 () {54 vars in 927 bytes} [Tue Sep 6 04:30:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 04:30:16.473900 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9146ade3-d551-401f-8c97-195d3ba0b7ce None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:30:16.474308 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9146ade3-d551-401f-8c97-195d3ba0b7ce None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:30:16.474451 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9146ade3-d551-401f-8c97-195d3ba0b7ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:30:16.541637 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-9146ade3-d551-401f-8c97-195d3ba0b7ce None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:30:16.542837 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-9146ade3-d551-401f-8c97-195d3ba0b7ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:30:16.637927 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 750/1480] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Sep 6 04:30:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 04:30:16.943526 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7d7e9a-183b-4673-bff2-e29b96034abb None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:30:16.945063 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-4e7d7e9a-183b-4673-bff2-e29b96034abb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:30:16.956503 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:30:17.111997 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7d7e9a-183b-4673-bff2-e29b96034abb None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:30:17.221822 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:30:17.263420 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d58af87-f48f-4eb1-b30e-df0a7251e861 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9db138b77afa4b068244b8ef715404fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b707733ce65c42edbc1fd95ad0a1c06c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:30:17.264075 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d58af87-f48f-4eb1-b30e-df0a7251e861 service nova] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:30:17.264376 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d58af87-f48f-4eb1-b30e-df0a7251e861 service nova] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:30:17.264657 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d58af87-f48f-4eb1-b30e-df0a7251e861 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:30:17.266325 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-1d58af87-f48f-4eb1-b30e-df0a7251e861 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:30:17.401586 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d58af87-f48f-4eb1-b30e-df0a7251e861 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:30:17.403192 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d58af87-f48f-4eb1-b30e-df0a7251e861 service nova] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:30:17.421484 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 731/1481] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Sep 6 04:30:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 478 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 04:30:19.543552 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6fe0b0-f548-4bac-863e-0c4a6a5eb92d None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:30:19.544122 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6fe0b0-f548-4bac-863e-0c4a6a5eb92d None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:30:19.544122 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6fe0b0-f548-4bac-863e-0c4a6a5eb92d None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:30:19.545494 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 751/1482] 10.222.0.30 () {54 vars in 927 bytes} [Tue Sep 6 04:30:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 04:30:19.551385 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e0c0eb-f093-4b3a-a1ef-35ee7b9e9372 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:30:19.551385 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e0c0eb-f093-4b3a-a1ef-35ee7b9e9372 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:30:19.552201 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e0c0eb-f093-4b3a-a1ef-35ee7b9e9372 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:30:19.617374 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-e8e0c0eb-f093-4b3a-a1ef-35ee7b9e9372 None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:30:19.618576 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-e8e0c0eb-f093-4b3a-a1ef-35ee7b9e9372 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:30:19.664071 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:30:19.714937 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 732/1483] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Sep 6 04:30:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 04:30:20.001212 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6fe0b0-f548-4bac-863e-0c4a6a5eb92d None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:30:20.002780 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-dd6fe0b0-f548-4bac-863e-0c4a6a5eb92d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:30:20.027026 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:30:20.106271 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6fe0b0-f548-4bac-863e-0c4a6a5eb92d None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:30:20.182929 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b971d50-b76d-40c2-8981-9adcd23d47d1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90492440f6a6471e8392361c098428aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b707733ce65c42edbc1fd95ad0a1c06c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:30:20.183542 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b971d50-b76d-40c2-8981-9adcd23d47d1 service neutron] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:30:20.183822 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b971d50-b76d-40c2-8981-9adcd23d47d1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:30:20.184092 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b971d50-b76d-40c2-8981-9adcd23d47d1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:30:20.185835 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-8b971d50-b76d-40c2-8981-9adcd23d47d1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:30:20.260236 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:30:20.294899 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b971d50-b76d-40c2-8981-9adcd23d47d1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:30:20.296350 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b971d50-b76d-40c2-8981-9adcd23d47d1 service neutron] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:30:20.315808 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 752/1484] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Sep 6 04:30:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 04:30:22.309524 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277618c0-fd0c-4afa-9ab2-3b5c928aa0e7 None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:30:22.310081 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277618c0-fd0c-4afa-9ab2-3b5c928aa0e7 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:30:22.310081 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277618c0-fd0c-4afa-9ab2-3b5c928aa0e7 None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:30:22.311443 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 733/1485] 10.222.0.30 () {54 vars in 927 bytes} [Tue Sep 6 04:30:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 04:30:22.317339 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cb4c0c-99e8-4db5-8a7f-618057447d90 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:30:22.317744 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cb4c0c-99e8-4db5-8a7f-618057447d90 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:30:22.317875 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cb4c0c-99e8-4db5-8a7f-618057447d90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:30:22.383984 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-f8cb4c0c-99e8-4db5-8a7f-618057447d90 None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:30:22.385707 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-f8cb4c0c-99e8-4db5-8a7f-618057447d90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:30:22.487437 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 753/1486] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Sep 6 04:30:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 04:30:22.777173 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277618c0-fd0c-4afa-9ab2-3b5c928aa0e7 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:30:22.778750 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-277618c0-fd0c-4afa-9ab2-3b5c928aa0e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:30:22.881778 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277618c0-fd0c-4afa-9ab2-3b5c928aa0e7 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:30:22.952415 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:30:22.954216 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eae5f0d-88d3-450e-8e30-5a3a87c773ca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90492440f6a6471e8392361c098428aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b707733ce65c42edbc1fd95ad0a1c06c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:30:22.954829 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eae5f0d-88d3-450e-8e30-5a3a87c773ca service neutron] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:30:22.955104 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eae5f0d-88d3-450e-8e30-5a3a87c773ca service neutron] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:30:22.955373 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eae5f0d-88d3-450e-8e30-5a3a87c773ca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:30:22.957691 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-3eae5f0d-88d3-450e-8e30-5a3a87c773ca service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:30:23.063958 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eae5f0d-88d3-450e-8e30-5a3a87c773ca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:30:23.065401 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eae5f0d-88d3-450e-8e30-5a3a87c773ca service neutron] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:30:23.084304 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 734/1487] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Sep 6 04:30:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 04:30:25.123077 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11b3fd1-657a-41e5-8b8a-60a7a459b719 None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:30:25.123627 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11b3fd1-657a-41e5-8b8a-60a7a459b719 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:30:25.123627 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11b3fd1-657a-41e5-8b8a-60a7a459b719 None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:30:25.124985 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 754/1488] 10.222.0.30 () {54 vars in 927 bytes} [Tue Sep 6 04:30:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 04:30:25.130552 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba542cf-120b-4838-a864-c0629a9f3664 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:30:25.130989 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba542cf-120b-4838-a864-c0629a9f3664 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:30:25.131103 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba542cf-120b-4838-a864-c0629a9f3664 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:30:25.198437 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-aba542cf-120b-4838-a864-c0629a9f3664 None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:30:25.199586 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-aba542cf-120b-4838-a864-c0629a9f3664 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:30:25.253628 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:30:25.298650 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 735/1489] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Sep 6 04:30:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 04:30:25.592970 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11b3fd1-657a-41e5-8b8a-60a7a459b719 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:30:25.594558 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-e11b3fd1-657a-41e5-8b8a-60a7a459b719 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:30:25.630574 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:30:25.698701 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11b3fd1-657a-41e5-8b8a-60a7a459b719 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:30:25.780445 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae5c2f3-5a63-433d-9d15-b20d4a0dec04 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90492440f6a6471e8392361c098428aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b707733ce65c42edbc1fd95ad0a1c06c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:30:25.781050 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae5c2f3-5a63-433d-9d15-b20d4a0dec04 service neutron] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:30:25.781345 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae5c2f3-5a63-433d-9d15-b20d4a0dec04 service neutron] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:30:25.781619 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae5c2f3-5a63-433d-9d15-b20d4a0dec04 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:30:25.783204 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-7ae5c2f3-5a63-433d-9d15-b20d4a0dec04 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:30:25.862842 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:30:25.892582 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae5c2f3-5a63-433d-9d15-b20d4a0dec04 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:30:25.894060 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae5c2f3-5a63-433d-9d15-b20d4a0dec04 service neutron] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:30:25.912970 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 755/1490] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Sep 6 04:30:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 04:30:28.267127 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c0bfb9-b303-45bb-be9a-17f8761c723b None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:30:28.267777 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c0bfb9-b303-45bb-be9a-17f8761c723b None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:30:28.267777 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c0bfb9-b303-45bb-be9a-17f8761c723b None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:30:28.269019 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 736/1491] 10.222.0.30 () {54 vars in 927 bytes} [Tue Sep 6 04:30:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 04:30:28.274645 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70125f0a-6904-405f-88de-a7432eddfff8 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:30:28.275136 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70125f0a-6904-405f-88de-a7432eddfff8 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:30:28.275511 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70125f0a-6904-405f-88de-a7432eddfff8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:30:28.341894 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-70125f0a-6904-405f-88de-a7432eddfff8 None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:30:28.343096 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-70125f0a-6904-405f-88de-a7432eddfff8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:30:28.439298 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 756/1492] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Sep 6 04:30:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 04:30:28.736856 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c0bfb9-b303-45bb-be9a-17f8761c723b None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:30:28.738674 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-42c0bfb9-b303-45bb-be9a-17f8761c723b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:30:28.841389 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c0bfb9-b303-45bb-be9a-17f8761c723b None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:30:28.922773 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e351292f-5457-4ae6-883e-18285d1414fe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90492440f6a6471e8392361c098428aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b707733ce65c42edbc1fd95ad0a1c06c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:30:28.923452 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e351292f-5457-4ae6-883e-18285d1414fe service neutron] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:30:28.923732 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e351292f-5457-4ae6-883e-18285d1414fe service neutron] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:30:28.924006 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e351292f-5457-4ae6-883e-18285d1414fe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:30:28.925714 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-e351292f-5457-4ae6-883e-18285d1414fe service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:30:28.935269 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:30:29.036863 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e351292f-5457-4ae6-883e-18285d1414fe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:30:29.038437 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e351292f-5457-4ae6-883e-18285d1414fe service neutron] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:30:29.056524 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 737/1493] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Sep 6 04:30:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 04:30:38.437455 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=82 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:30:39.054672 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=50 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:31:52.603604 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66151202-35a9-4778-8cf6-14bb72205103 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:31:52.604386 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66151202-35a9-4778-8cf6-14bb72205103 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:31:52.604386 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66151202-35a9-4778-8cf6-14bb72205103 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:31:52.678999 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-66151202-35a9-4778-8cf6-14bb72205103 None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:31:52.680140 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-66151202-35a9-4778-8cf6-14bb72205103 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:31:52.776675 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 757/1494] 10.222.0.30 () {58 vars in 958 bytes} [Tue Sep 6 04:31:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 04:31:52.784759 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7eb775-c6ba-4b18-bb8a-6e43a9611c68 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:31:52.785261 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7eb775-c6ba-4b18-bb8a-6e43a9611c68 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:31:52.785390 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7eb775-c6ba-4b18-bb8a-6e43a9611c68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:31:52.860563 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-8e7eb775-c6ba-4b18-bb8a-6e43a9611c68 None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:31:52.861822 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-8e7eb775-c6ba-4b18-bb8a-6e43a9611c68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:31:52.963219 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 738/1495] 10.222.0.30 () {58 vars in 958 bytes} [Tue Sep 6 04:31:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 04:31:52.968471 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66151202-35a9-4778-8cf6-14bb72205103 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:31:52.969221 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-66151202-35a9-4778-8cf6-14bb72205103 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:31:52.980679 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66151202-35a9-4778-8cf6-14bb72205103 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:31:53.041186 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:31:53.074444 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db14997-b60d-43ad-92cf-6e70a791aa4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:31:53.075053 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db14997-b60d-43ad-92cf-6e70a791aa4c admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:31:53.075333 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db14997-b60d-43ad-92cf-6e70a791aa4c admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:31:53.075604 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db14997-b60d-43ad-92cf-6e70a791aa4c admin admin] PATH_INFO: `/v3/domains` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:31:53.076250 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db14997-b60d-43ad-92cf-6e70a791aa4c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:31:53.076551 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db14997-b60d-43ad-92cf-6e70a791aa4c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:31:53.077612 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: WARNING py.warnings [None req-7db14997-b60d-43ad-92cf-6e70a791aa4c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 06 04:31:53.077612 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: warnings.warn(msg) Sep 06 04:31:53.077612 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: Sep 06 04:31:53.078501 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db14997-b60d-43ad-92cf-6e70a791aa4c admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:31:53.090437 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 758/1496] 10.222.0.30 () {58 vars in 1153 bytes} [Tue Sep 6 04:31:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:31:53.098913 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7eb775-c6ba-4b18-bb8a-6e43a9611c68 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:31:53.108557 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7eb775-c6ba-4b18-bb8a-6e43a9611c68 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:31:53.168753 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:31:53.200940 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7169c5c-7ac2-4549-bb49-f3460c299cc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:31:53.201574 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7169c5c-7ac2-4549-bb49-f3460c299cc6 admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:31:53.202043 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7169c5c-7ac2-4549-bb49-f3460c299cc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:31:53.202537 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7169c5c-7ac2-4549-bb49-f3460c299cc6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:31:53.203487 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7169c5c-7ac2-4549-bb49-f3460c299cc6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:31:53.205228 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: WARNING py.warnings [None req-e7169c5c-7ac2-4549-bb49-f3460c299cc6 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 06 04:31:53.205228 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: warnings.warn(msg) Sep 06 04:31:53.205228 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: Sep 06 04:31:53.206484 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7169c5c-7ac2-4549-bb49-f3460c299cc6 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:31:53.351915 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 739/1497] 10.222.0.30 () {60 vars in 1153 bytes} [Tue Sep 6 04:31:53 2022] POST /identity/v3/projects => generated 379 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 04:31:53.356912 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db14997-b60d-43ad-92cf-6e70a791aa4c admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:31:53.358464 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-7db14997-b60d-43ad-92cf-6e70a791aa4c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:31:53.392575 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db14997-b60d-43ad-92cf-6e70a791aa4c admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:31:53.485596 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f7a309-fe10-4646-896e-a96c150666af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:31:53.486209 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f7a309-fe10-4646-896e-a96c150666af admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:31:53.486480 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f7a309-fe10-4646-896e-a96c150666af admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:31:53.486741 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f7a309-fe10-4646-896e-a96c150666af admin admin] PATH_INFO: `/v3/users` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:31:53.487371 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f7a309-fe10-4646-896e-a96c150666af admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:31:53.488371 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: WARNING py.warnings [None req-f0f7a309-fe10-4646-896e-a96c150666af admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 06 04:31:53.488371 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: warnings.warn(msg) Sep 06 04:31:53.488371 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: Sep 06 04:31:53.489610 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f7a309-fe10-4646-896e-a96c150666af admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:31:53.593142 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 759/1498] 10.222.0.30 () {60 vars in 1144 bytes} [Tue Sep 6 04:31:53 2022] POST /identity/v3/users => generated 290 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 04:31:53.597393 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7169c5c-7ac2-4549-bb49-f3460c299cc6 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:31:53.598906 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-e7169c5c-7ac2-4549-bb49-f3460c299cc6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:31:53.661699 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7169c5c-7ac2-4549-bb49-f3460c299cc6 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:31:53.746607 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:31:53.756762 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7488cb4-7cff-4096-bead-10ffa82b4d70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:31:53.757409 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7488cb4-7cff-4096-bead-10ffa82b4d70 admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:31:53.757681 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7488cb4-7cff-4096-bead-10ffa82b4d70 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:31:53.757951 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7488cb4-7cff-4096-bead-10ffa82b4d70 admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:31:53.758591 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7488cb4-7cff-4096-bead-10ffa82b4d70 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:31:53.760148 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: WARNING py.warnings [None req-f7488cb4-7cff-4096-bead-10ffa82b4d70 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 06 04:31:53.760148 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: warnings.warn(msg) Sep 06 04:31:53.760148 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: Sep 06 04:31:53.761034 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7488cb4-7cff-4096-bead-10ffa82b4d70 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:31:53.773429 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 740/1499] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Sep 6 04:31:53 2022] GET /identity/v3/roles => generated 1420 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 04:31:53.777623 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f7a309-fe10-4646-896e-a96c150666af admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:31:53.779140 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-f0f7a309-fe10-4646-896e-a96c150666af admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:31:53.790829 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f7a309-fe10-4646-896e-a96c150666af admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:31:53.816176 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:31:53.885477 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460601a1-b7dc-42ce-b941-246602ab5cd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:31:53.886083 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460601a1-b7dc-42ce-b941-246602ab5cd5 admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:31:53.886356 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460601a1-b7dc-42ce-b941-246602ab5cd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:31:53.886620 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460601a1-b7dc-42ce-b941-246602ab5cd5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:31:53.887255 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460601a1-b7dc-42ce-b941-246602ab5cd5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:31:53.888261 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: WARNING py.warnings [None req-460601a1-b7dc-42ce-b941-246602ab5cd5 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 06 04:31:53.888261 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: warnings.warn(msg) Sep 06 04:31:53.888261 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: Sep 06 04:31:53.889139 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460601a1-b7dc-42ce-b941-246602ab5cd5 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:31:53.902200 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 760/1500] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Sep 6 04:31:53 2022] GET /identity/v3/roles => generated 1420 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 04:31:53.906407 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7488cb4-7cff-4096-bead-10ffa82b4d70 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:31:53.907934 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-f7488cb4-7cff-4096-bead-10ffa82b4d70 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:31:53.942040 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7488cb4-7cff-4096-bead-10ffa82b4d70 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:31:54.036773 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cdc1078-adb8-44a2-88ad-0392c24b0d53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:31:54.037453 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdc1078-adb8-44a2-88ad-0392c24b0d53 admin admin] REQUEST_METHOD: `PUT` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:31:54.037729 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdc1078-adb8-44a2-88ad-0392c24b0d53 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:31:54.037992 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdc1078-adb8-44a2-88ad-0392c24b0d53 admin admin] PATH_INFO: `/v3/projects/bf31402ea3fd42b482325b338980019b/users/99cdfd01dcaa4e14b58098a7efb59c1b/roles/29607b2de39f40d68bb6ceef64b8a732` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:31:54.068326 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cdc1078-adb8-44a2-88ad-0392c24b0d53 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bf31402ea3fd42b482325b338980019b, user_id=99cdfd01dcaa4e14b58098a7efb59c1b, role_id=29607b2de39f40d68bb6ceef64b8a732)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:31:54.068326 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: WARNING py.warnings [None req-0cdc1078-adb8-44a2-88ad-0392c24b0d53 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 06 04:31:54.068326 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: warnings.warn(msg) Sep 06 04:31:54.068326 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: Sep 06 04:31:54.068326 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cdc1078-adb8-44a2-88ad-0392c24b0d53 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:31:54.180166 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 741/1501] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Sep 6 04:31:53 2022] PUT /identity/v3/projects/bf31402ea3fd42b482325b338980019b/users/99cdfd01dcaa4e14b58098a7efb59c1b/roles/29607b2de39f40d68bb6ceef64b8a732 => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 04:31:54.192519 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e396fc-e427-4fb2-95b7-80cd7edac9cf None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:31:54.192744 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e396fc-e427-4fb2-95b7-80cd7edac9cf None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:31:54.193018 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e396fc-e427-4fb2-95b7-80cd7edac9cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:31:54.249372 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-71e396fc-e427-4fb2-95b7-80cd7edac9cf None None] MFA Rules not processed for user `99cdfd01dcaa4e14b58098a7efb59c1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:31:54.250583 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-71e396fc-e427-4fb2-95b7-80cd7edac9cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:31:54.327846 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 761/1502] 10.222.0.30 () {58 vars in 958 bytes} [Tue Sep 6 04:31:54 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 04:31:54.341982 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed39fa86-830a-45c1-aac2-f74e8b2d2a4c None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:31:54.342266 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed39fa86-830a-45c1-aac2-f74e8b2d2a4c None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:31:54.342566 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed39fa86-830a-45c1-aac2-f74e8b2d2a4c None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:31:54.343904 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 742/1503] 10.222.0.30 () {54 vars in 964 bytes} [Tue Sep 6 04:31:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 04:31:54.348784 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2febe28-10fe-4426-a55d-1b143e35bc07 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:31:54.349075 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2febe28-10fe-4426-a55d-1b143e35bc07 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:31:54.349384 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2febe28-10fe-4426-a55d-1b143e35bc07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:31:54.381380 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:31:54.427684 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-b2febe28-10fe-4426-a55d-1b143e35bc07 None None] MFA Rules not processed for user `d09b5ab675634a2eb4b03755743f7d0d`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:31:54.428899 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-b2febe28-10fe-4426-a55d-1b143e35bc07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:31:54.504529 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 762/1504] 10.222.0.30 () {58 vars in 1061 bytes} [Tue Sep 6 04:31:54 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 04:31:54.512705 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed39fa86-830a-45c1-aac2-f74e8b2d2a4c None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:31:54.514282 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-ed39fa86-830a-45c1-aac2-f74e8b2d2a4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:31:54.564662 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:31:54.618414 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed39fa86-830a-45c1-aac2-f74e8b2d2a4c None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:31:54.690986 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c67a31-dd39-415c-84ce-f56bbe8b2bfb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd09b5ab675634a2eb4b03755743f7d0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b707733ce65c42edbc1fd95ad0a1c06c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:31:54.691600 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c67a31-dd39-415c-84ce-f56bbe8b2bfb service cinder] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:31:54.691882 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c67a31-dd39-415c-84ce-f56bbe8b2bfb service cinder] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:31:54.692156 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c67a31-dd39-415c-84ce-f56bbe8b2bfb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:31:54.693794 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-32c67a31-dd39-415c-84ce-f56bbe8b2bfb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:31:54.793069 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c67a31-dd39-415c-84ce-f56bbe8b2bfb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:31:54.794550 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c67a31-dd39-415c-84ce-f56bbe8b2bfb service cinder] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:31:54.798048 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:31:54.812116 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 743/1505] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Sep 6 04:31:54 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 06 04:31:54.873374 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff40e01-5f7c-46e2-9554-3f96dd799e75 None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:31:54.873831 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff40e01-5f7c-46e2-9554-3f96dd799e75 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:31:54.873944 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff40e01-5f7c-46e2-9554-3f96dd799e75 None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:31:54.875278 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 763/1506] 10.222.0.30 () {52 vars in 830 bytes} [Tue Sep 6 04:31:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 04:31:54.878837 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c67a31-dd39-415c-84ce-f56bbe8b2bfb service cinder] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:31:54.880055 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-32c67a31-dd39-415c-84ce-f56bbe8b2bfb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:31:54.984740 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c67a31-dd39-415c-84ce-f56bbe8b2bfb service cinder] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:31:55.075973 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45cc7caf-0f7c-40bd-8b5f-8f9a2d568c53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:31:55.076614 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cc7caf-0f7c-40bd-8b5f-8f9a2d568c53 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:31:55.076893 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cc7caf-0f7c-40bd-8b5f-8f9a2d568c53 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:31:55.077164 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cc7caf-0f7c-40bd-8b5f-8f9a2d568c53 admin admin] PATH_INFO: `/v3/users/99cdfd01dcaa4e14b58098a7efb59c1b` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:31:55.089584 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45cc7caf-0f7c-40bd-8b5f-8f9a2d568c53 admin admin] RBAC: Authorizing `identity:delete_user(user_id=99cdfd01dcaa4e14b58098a7efb59c1b)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:31:55.090634 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: WARNING py.warnings [None req-45cc7caf-0f7c-40bd-8b5f-8f9a2d568c53 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 06 04:31:55.090634 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: warnings.warn(msg) Sep 06 04:31:55.090634 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: Sep 06 04:31:55.091840 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45cc7caf-0f7c-40bd-8b5f-8f9a2d568c53 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:31:55.101260 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:31:55.262807 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-45cc7caf-0f7c-40bd-8b5f-8f9a2d568c53 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '99cdfd01dcaa4e14b58098a7efb59c1b'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:31:55.327757 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-45cc7caf-0f7c-40bd-8b5f-8f9a2d568c53 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '99cdfd01dcaa4e14b58098a7efb59c1b'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:31:55.328218 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-45cc7caf-0f7c-40bd-8b5f-8f9a2d568c53 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '99cdfd01dcaa4e14b58098a7efb59c1b'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:31:55.342833 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-45cc7caf-0f7c-40bd-8b5f-8f9a2d568c53 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '99cdfd01dcaa4e14b58098a7efb59c1b'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:31:55.365608 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-45cc7caf-0f7c-40bd-8b5f-8f9a2d568c53 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '99cdfd01dcaa4e14b58098a7efb59c1b'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:31:55.372057 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 744/1507] 10.222.0.30 () {58 vars in 1224 bytes} [Tue Sep 6 04:31:54 2022] DELETE /identity/v3/users/99cdfd01dcaa4e14b58098a7efb59c1b => generated 0 bytes in 494 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 04:31:55.376510 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edfc1be-9b1f-4535-a66b-186adec5e697 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:31:55.376796 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edfc1be-9b1f-4535-a66b-186adec5e697 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:31:55.377068 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edfc1be-9b1f-4535-a66b-186adec5e697 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:31:55.434261 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-4edfc1be-9b1f-4535-a66b-186adec5e697 None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:31:55.435455 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-4edfc1be-9b1f-4535-a66b-186adec5e697 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:31:55.510257 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:31:55.522656 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 764/1508] 10.222.0.30 () {58 vars in 958 bytes} [Tue Sep 6 04:31:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 04:31:55.532940 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45cc7caf-0f7c-40bd-8b5f-8f9a2d568c53 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:31:55.534226 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-45cc7caf-0f7c-40bd-8b5f-8f9a2d568c53 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:31:55.631434 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45cc7caf-0f7c-40bd-8b5f-8f9a2d568c53 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:31:55.688182 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:31:55.708633 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d9a774-a09c-4955-9ee8-6aa141d63b1f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90492440f6a6471e8392361c098428aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b707733ce65c42edbc1fd95ad0a1c06c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:31:55.709279 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d9a774-a09c-4955-9ee8-6aa141d63b1f service neutron] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:31:55.709592 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d9a774-a09c-4955-9ee8-6aa141d63b1f service neutron] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:31:55.709867 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d9a774-a09c-4955-9ee8-6aa141d63b1f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:31:55.711470 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-10d9a774-a09c-4955-9ee8-6aa141d63b1f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:31:55.823670 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d9a774-a09c-4955-9ee8-6aa141d63b1f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:31:55.825101 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d9a774-a09c-4955-9ee8-6aa141d63b1f service neutron] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:31:55.843542 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 745/1509] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Sep 6 04:31:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 04:31:56.317995 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4edfc1be-9b1f-4535-a66b-186adec5e697 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:31:56.319251 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-4edfc1be-9b1f-4535-a66b-186adec5e697 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:31:56.331517 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4edfc1be-9b1f-4535-a66b-186adec5e697 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:31:56.432147 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33bf5622-b1fd-41f5-9e44-66004b38d38f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:31:56.435154 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bf5622-b1fd-41f5-9e44-66004b38d38f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:31:56.435154 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bf5622-b1fd-41f5-9e44-66004b38d38f admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:31:56.435154 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bf5622-b1fd-41f5-9e44-66004b38d38f admin admin] PATH_INFO: `/v3/projects/bf31402ea3fd42b482325b338980019b` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:31:56.443261 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33bf5622-b1fd-41f5-9e44-66004b38d38f admin admin] RBAC: Authorizing `identity:delete_project(project_id=bf31402ea3fd42b482325b338980019b)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:31:56.444293 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: WARNING py.warnings [None req-33bf5622-b1fd-41f5-9e44-66004b38d38f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 06 04:31:56.444293 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: warnings.warn(msg) Sep 06 04:31:56.444293 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: Sep 06 04:31:56.445508 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33bf5622-b1fd-41f5-9e44-66004b38d38f admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:31:56.562272 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-33bf5622-b1fd-41f5-9e44-66004b38d38f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bf31402ea3fd42b482325b338980019b', 'request_id': 'req-33bf5622-b1fd-41f5-9e44-66004b38d38f'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:31:56.568312 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-33bf5622-b1fd-41f5-9e44-66004b38d38f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bf31402ea3fd42b482325b338980019b', 'request_id': 'req-33bf5622-b1fd-41f5-9e44-66004b38d38f'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:31:56.648546 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-33bf5622-b1fd-41f5-9e44-66004b38d38f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bf31402ea3fd42b482325b338980019b', 'request_id': 'req-33bf5622-b1fd-41f5-9e44-66004b38d38f'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:31:56.681553 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-33bf5622-b1fd-41f5-9e44-66004b38d38f admin admin] The token cache is being invalidate because project bf31402ea3fd42b482325b338980019b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81698) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 04:31:56.681855 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-33bf5622-b1fd-41f5-9e44-66004b38d38f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:31:56.682382 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-33bf5622-b1fd-41f5-9e44-66004b38d38f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:31:56.683665 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 765/1510] 10.222.0.30 () {58 vars in 1233 bytes} [Tue Sep 6 04:31:56 2022] DELETE /identity/v3/projects/bf31402ea3fd42b482325b338980019b => generated 0 bytes in 366 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 04:32:05.841289 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=62 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:32:06.678963 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:33:20.877214 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437bad96-bf28-4677-b4df-ce4c3b7805ca None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:33:20.877826 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437bad96-bf28-4677-b4df-ce4c3b7805ca None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:33:20.877826 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437bad96-bf28-4677-b4df-ce4c3b7805ca None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:33:20.879141 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 746/1511] 127.0.0.1 () {54 vars in 901 bytes} [Tue Sep 6 04:33:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 04:33:20.884843 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afab513-202a-4b1e-b4be-46b9ce6ec58a None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:33:20.885241 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afab513-202a-4b1e-b4be-46b9ce6ec58a None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:33:20.885412 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afab513-202a-4b1e-b4be-46b9ce6ec58a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:33:20.965452 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-5afab513-202a-4b1e-b4be-46b9ce6ec58a None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:33:20.966653 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-5afab513-202a-4b1e-b4be-46b9ce6ec58a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:33:21.069645 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 766/1512] 127.0.0.1 () {58 vars in 998 bytes} [Tue Sep 6 04:33:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 04:33:21.223932 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437bad96-bf28-4677-b4df-ce4c3b7805ca None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:33:21.226092 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-437bad96-bf28-4677-b4df-ce4c3b7805ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:33:21.350987 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437bad96-bf28-4677-b4df-ce4c3b7805ca None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:33:21.356830 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968c9ea0-253c-485f-935e-6cba6189f265 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9db138b77afa4b068244b8ef715404fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b707733ce65c42edbc1fd95ad0a1c06c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:33:21.357489 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968c9ea0-253c-485f-935e-6cba6189f265 service nova] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:33:21.357773 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968c9ea0-253c-485f-935e-6cba6189f265 service nova] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:33:21.358046 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968c9ea0-253c-485f-935e-6cba6189f265 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:33:21.366926 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968c9ea0-253c-485f-935e-6cba6189f265 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:33:21.368899 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968c9ea0-253c-485f-935e-6cba6189f265 service nova] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:33:21.378423 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 747/1513] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Sep 6 04:33:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 06 04:33:22.219923 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec86b16-09de-45d8-9c2f-2d2284f9fca1 None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:33:22.220460 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec86b16-09de-45d8-9c2f-2d2284f9fca1 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:33:22.220460 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec86b16-09de-45d8-9c2f-2d2284f9fca1 None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:33:22.221855 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 767/1514] 127.0.0.1 () {54 vars in 901 bytes} [Tue Sep 6 04:33:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 04:33:22.226723 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23c7f5f-6e8c-442d-bb96-7f387d0ce16f None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:33:22.227152 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23c7f5f-6e8c-442d-bb96-7f387d0ce16f None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:33:22.227259 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23c7f5f-6e8c-442d-bb96-7f387d0ce16f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:33:22.246742 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-b23c7f5f-6e8c-442d-bb96-7f387d0ce16f None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:33:22.247950 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-b23c7f5f-6e8c-442d-bb96-7f387d0ce16f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:33:22.255830 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 748/1515] 127.0.0.1 () {58 vars in 998 bytes} [Tue Sep 6 04:33:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 04:33:22.409591 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec86b16-09de-45d8-9c2f-2d2284f9fca1 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:33:22.424384 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec86b16-09de-45d8-9c2f-2d2284f9fca1 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:33:22.429657 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96c72c4-e12f-474e-815d-110f1d5d954a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9db138b77afa4b068244b8ef715404fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b707733ce65c42edbc1fd95ad0a1c06c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:33:22.430248 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96c72c4-e12f-474e-815d-110f1d5d954a service nova] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:33:22.431057 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96c72c4-e12f-474e-815d-110f1d5d954a service nova] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:33:22.431417 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96c72c4-e12f-474e-815d-110f1d5d954a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:33:22.439447 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96c72c4-e12f-474e-815d-110f1d5d954a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:33:22.441022 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96c72c4-e12f-474e-815d-110f1d5d954a service nova] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:33:22.450993 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 768/1516] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Sep 6 04:33:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 04:33:22.866425 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ad1413-87cd-4063-a76e-1d8338882468 None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:33:22.866707 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ad1413-87cd-4063-a76e-1d8338882468 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:33:22.866981 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ad1413-87cd-4063-a76e-1d8338882468 None None] PATH_INFO: `/v3` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:33:22.868895 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 749/1517] 10.222.0.46 () {54 vars in 925 bytes} [Tue Sep 6 04:33:22 2022] GET /identity/v3 => generated 255 bytes in 4 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:33:22.891510 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e86071f-1d51-4cc0-9a42-1c8a3a48799f None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:33:22.891793 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e86071f-1d51-4cc0-9a42-1c8a3a48799f None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:33:22.892087 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e86071f-1d51-4cc0-9a42-1c8a3a48799f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:33:22.950508 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-4e86071f-1d51-4cc0-9a42-1c8a3a48799f None None] MFA Rules not processed for user `1d4f2e4dbd41441d84f03ddb30191e7a`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:33:22.951694 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-4e86071f-1d51-4cc0-9a42-1c8a3a48799f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:33:22.967713 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:33:23.021414 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 769/1518] 10.222.0.46 () {58 vars in 1015 bytes} [Tue Sep 6 04:33:22 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 04:33:32.241776 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=56 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:33:32.339747 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f6a9ac-83ca-4b46-b67d-65b1964fca94 None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:33:32.340045 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f6a9ac-83ca-4b46-b67d-65b1964fca94 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:33:32.340298 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f6a9ac-83ca-4b46-b67d-65b1964fca94 None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:33:32.341842 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 750/1519] 10.222.0.30 () {54 vars in 961 bytes} [Tue Sep 6 04:33:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 04:33:32.347313 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2870fbd1-1a93-497b-9a4a-ebfd232a750b None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:33:32.347809 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2870fbd1-1a93-497b-9a4a-ebfd232a750b None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:33:32.347956 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2870fbd1-1a93-497b-9a4a-ebfd232a750b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:33:32.368614 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-2870fbd1-1a93-497b-9a4a-ebfd232a750b None None] MFA Rules not processed for user `1d4f2e4dbd41441d84f03ddb30191e7a`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:33:32.369779 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-2870fbd1-1a93-497b-9a4a-ebfd232a750b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:33:32.378317 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 770/1520] 10.222.0.30 () {58 vars in 1058 bytes} [Tue Sep 6 04:33:32 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 04:33:32.386634 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f6a9ac-83ca-4b46-b67d-65b1964fca94 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:33:32.400681 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f6a9ac-83ca-4b46-b67d-65b1964fca94 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:33:32.405596 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07162b6-1f41-4e20-8114-b01059435d7a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d4f2e4dbd41441d84f03ddb30191e7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b707733ce65c42edbc1fd95ad0a1c06c', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:33:32.406229 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07162b6-1f41-4e20-8114-b01059435d7a service placement] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:33:32.406511 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07162b6-1f41-4e20-8114-b01059435d7a service placement] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:33:32.406809 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07162b6-1f41-4e20-8114-b01059435d7a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:33:32.415639 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07162b6-1f41-4e20-8114-b01059435d7a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:33:32.417128 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07162b6-1f41-4e20-8114-b01059435d7a service placement] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:33:32.426822 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 751/1521] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Sep 6 04:33:32 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 04:33:33.731072 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b4cf67-3244-4d00-87ff-ef60ff805c4c None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:33:33.731625 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b4cf67-3244-4d00-87ff-ef60ff805c4c None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:33:33.731625 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b4cf67-3244-4d00-87ff-ef60ff805c4c None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:33:33.732886 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 771/1522] 127.0.0.1 () {54 vars in 901 bytes} [Tue Sep 6 04:33:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 04:33:33.737804 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b47bb1b-b391-49b7-b58e-61a8978f0d89 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:33:33.738205 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b47bb1b-b391-49b7-b58e-61a8978f0d89 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:33:33.738351 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b47bb1b-b391-49b7-b58e-61a8978f0d89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:33:33.758024 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-9b47bb1b-b391-49b7-b58e-61a8978f0d89 None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:33:33.759906 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-9b47bb1b-b391-49b7-b58e-61a8978f0d89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:33:33.767673 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 752/1523] 127.0.0.1 () {58 vars in 998 bytes} [Tue Sep 6 04:33:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 04:33:33.921825 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b4cf67-3244-4d00-87ff-ef60ff805c4c None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:33:33.936071 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b4cf67-3244-4d00-87ff-ef60ff805c4c None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:33:33.941584 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04db6572-14ee-489d-8081-45b030b2b50e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9db138b77afa4b068244b8ef715404fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b707733ce65c42edbc1fd95ad0a1c06c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:33:33.944533 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04db6572-14ee-489d-8081-45b030b2b50e service nova] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:33:33.944533 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04db6572-14ee-489d-8081-45b030b2b50e service nova] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:33:33.944533 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04db6572-14ee-489d-8081-45b030b2b50e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:33:33.951439 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04db6572-14ee-489d-8081-45b030b2b50e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:33:33.953030 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04db6572-14ee-489d-8081-45b030b2b50e service nova] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:33:33.963162 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 772/1524] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Sep 6 04:33:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 04:33:34.798351 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423a3091-f726-483d-ba56-b7dce4235bfa None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:33:34.798897 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423a3091-f726-483d-ba56-b7dce4235bfa None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:33:34.798897 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423a3091-f726-483d-ba56-b7dce4235bfa None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:33:34.800357 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 753/1525] 127.0.0.1 () {54 vars in 901 bytes} [Tue Sep 6 04:33:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 04:33:34.807213 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7bcd4b-870b-4e81-a4b2-c51d7bc9938a None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:33:34.807724 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7bcd4b-870b-4e81-a4b2-c51d7bc9938a None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:33:34.807724 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7bcd4b-870b-4e81-a4b2-c51d7bc9938a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:33:34.828197 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-3c7bcd4b-870b-4e81-a4b2-c51d7bc9938a None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:33:34.829415 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-3c7bcd4b-870b-4e81-a4b2-c51d7bc9938a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:33:34.837466 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 773/1526] 127.0.0.1 () {58 vars in 998 bytes} [Tue Sep 6 04:33:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 04:33:34.988082 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423a3091-f726-483d-ba56-b7dce4235bfa None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:33:35.003046 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423a3091-f726-483d-ba56-b7dce4235bfa None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:33:35.008491 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785828e5-06ca-4ee2-a6ff-cffec205895a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9db138b77afa4b068244b8ef715404fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b707733ce65c42edbc1fd95ad0a1c06c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:33:35.009161 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785828e5-06ca-4ee2-a6ff-cffec205895a service nova] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:33:35.009478 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785828e5-06ca-4ee2-a6ff-cffec205895a service nova] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:33:35.009753 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785828e5-06ca-4ee2-a6ff-cffec205895a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:33:35.017720 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785828e5-06ca-4ee2-a6ff-cffec205895a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:33:35.019335 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785828e5-06ca-4ee2-a6ff-cffec205895a service nova] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:33:35.029612 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 754/1527] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Sep 6 04:33:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 04:33:38.844634 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fe457c-55e3-4a09-952c-bb8547fcf411 None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:33:38.845189 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fe457c-55e3-4a09-952c-bb8547fcf411 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:33:38.845189 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fe457c-55e3-4a09-952c-bb8547fcf411 None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:33:38.846625 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 774/1528] 127.0.0.1 () {54 vars in 917 bytes} [Tue Sep 6 04:33:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 04:33:38.851998 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa058122-375c-4b91-9d32-151b81f5c526 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:33:38.852446 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa058122-375c-4b91-9d32-151b81f5c526 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:33:38.852570 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa058122-375c-4b91-9d32-151b81f5c526 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:33:38.871996 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-fa058122-375c-4b91-9d32-151b81f5c526 None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:33:38.873192 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-fa058122-375c-4b91-9d32-151b81f5c526 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:33:38.880888 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 755/1529] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Sep 6 04:33:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 04:33:39.166729 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5fe457c-55e3-4a09-952c-bb8547fcf411 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:33:39.168465 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-b5fe457c-55e3-4a09-952c-bb8547fcf411 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:33:39.247923 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5fe457c-55e3-4a09-952c-bb8547fcf411 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:33:39.251508 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b8251a-58ae-4bec-9909-8c9c3f35f2a6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90492440f6a6471e8392361c098428aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b707733ce65c42edbc1fd95ad0a1c06c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:33:39.252134 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b8251a-58ae-4bec-9909-8c9c3f35f2a6 service neutron] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:33:39.252412 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b8251a-58ae-4bec-9909-8c9c3f35f2a6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:33:39.252681 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b8251a-58ae-4bec-9909-8c9c3f35f2a6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:33:39.262521 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b8251a-58ae-4bec-9909-8c9c3f35f2a6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:33:39.263952 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b8251a-58ae-4bec-9909-8c9c3f35f2a6 service neutron] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:33:39.273435 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 775/1530] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Sep 6 04:33:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 06 04:33:41.170246 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91de449a-537a-4312-b629-2a020e387fea None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:33:41.170805 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91de449a-537a-4312-b629-2a020e387fea None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:33:41.170805 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91de449a-537a-4312-b629-2a020e387fea None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:33:41.172137 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 756/1531] 127.0.0.1 () {54 vars in 917 bytes} [Tue Sep 6 04:33:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 04:33:41.177264 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f6cc64-bf00-4d35-ac96-b7251b87923b None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:33:41.177667 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f6cc64-bf00-4d35-ac96-b7251b87923b None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:33:41.177862 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f6cc64-bf00-4d35-ac96-b7251b87923b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:33:41.198346 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-62f6cc64-bf00-4d35-ac96-b7251b87923b None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:33:41.199535 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-62f6cc64-bf00-4d35-ac96-b7251b87923b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:33:41.207220 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 776/1532] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Sep 6 04:33:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 04:33:41.498445 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91de449a-537a-4312-b629-2a020e387fea None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:33:41.500144 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-91de449a-537a-4312-b629-2a020e387fea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:33:41.512982 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91de449a-537a-4312-b629-2a020e387fea None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:33:41.516507 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95b0e93-8f4e-4ff5-98bc-13118d1e1e0f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90492440f6a6471e8392361c098428aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b707733ce65c42edbc1fd95ad0a1c06c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:33:41.517102 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95b0e93-8f4e-4ff5-98bc-13118d1e1e0f service neutron] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:33:41.517418 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95b0e93-8f4e-4ff5-98bc-13118d1e1e0f service neutron] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:33:41.517690 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95b0e93-8f4e-4ff5-98bc-13118d1e1e0f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:33:41.526800 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95b0e93-8f4e-4ff5-98bc-13118d1e1e0f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:33:41.528224 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95b0e93-8f4e-4ff5-98bc-13118d1e1e0f service neutron] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:33:41.537960 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 757/1533] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Sep 6 04:33:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 06 04:33:44.858923 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33de1a4a-8e46-4283-829a-df110050f7ea None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:33:44.859484 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33de1a4a-8e46-4283-829a-df110050f7ea None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:33:44.859484 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33de1a4a-8e46-4283-829a-df110050f7ea None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:33:44.860863 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 777/1534] 127.0.0.1 () {54 vars in 917 bytes} [Tue Sep 6 04:33:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 04:33:44.865899 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8549363-d470-4fa7-b086-5301d2a617c5 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:33:44.866331 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8549363-d470-4fa7-b086-5301d2a617c5 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:33:44.866520 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8549363-d470-4fa7-b086-5301d2a617c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:33:44.885894 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-b8549363-d470-4fa7-b086-5301d2a617c5 None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:33:44.887102 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-b8549363-d470-4fa7-b086-5301d2a617c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:33:44.894915 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 758/1535] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Sep 6 04:33:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 04:33:45.452656 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33de1a4a-8e46-4283-829a-df110050f7ea None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:33:45.454490 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-33de1a4a-8e46-4283-829a-df110050f7ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:33:45.519371 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:33:45.533172 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33de1a4a-8e46-4283-829a-df110050f7ea None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:33:45.539376 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42dd1e42-6280-41f7-bf3c-3b24e1c1f86f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd09b5ab675634a2eb4b03755743f7d0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b707733ce65c42edbc1fd95ad0a1c06c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:33:45.539376 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42dd1e42-6280-41f7-bf3c-3b24e1c1f86f service cinder] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:33:45.539376 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42dd1e42-6280-41f7-bf3c-3b24e1c1f86f service cinder] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:33:45.539376 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42dd1e42-6280-41f7-bf3c-3b24e1c1f86f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:33:45.546880 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42dd1e42-6280-41f7-bf3c-3b24e1c1f86f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:33:45.548316 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42dd1e42-6280-41f7-bf3c-3b24e1c1f86f service cinder] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:33:45.558372 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 778/1536] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Sep 6 04:33:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 04:33:47.544719 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb10b9e0-0460-4c9a-afb3-c5c8bb595519 None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:33:47.545249 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb10b9e0-0460-4c9a-afb3-c5c8bb595519 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:33:47.545249 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb10b9e0-0460-4c9a-afb3-c5c8bb595519 None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:33:47.546626 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 759/1537] 127.0.0.1 () {54 vars in 917 bytes} [Tue Sep 6 04:33:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 04:33:47.551750 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3446f68d-f98a-43c1-b7fb-6b5af7671c31 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:33:47.552135 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3446f68d-f98a-43c1-b7fb-6b5af7671c31 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:33:47.552406 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3446f68d-f98a-43c1-b7fb-6b5af7671c31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:33:47.573150 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-3446f68d-f98a-43c1-b7fb-6b5af7671c31 None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:33:47.574446 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-3446f68d-f98a-43c1-b7fb-6b5af7671c31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:33:47.582668 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 779/1538] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Sep 6 04:33:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 04:33:48.151814 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb10b9e0-0460-4c9a-afb3-c5c8bb595519 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:33:48.153530 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-bb10b9e0-0460-4c9a-afb3-c5c8bb595519 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:33:48.167101 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb10b9e0-0460-4c9a-afb3-c5c8bb595519 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:33:48.170919 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e8f1cf-549a-4033-ab43-90363aa1cfbd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd09b5ab675634a2eb4b03755743f7d0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b707733ce65c42edbc1fd95ad0a1c06c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:33:48.171524 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e8f1cf-549a-4033-ab43-90363aa1cfbd service cinder] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:33:48.171802 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e8f1cf-549a-4033-ab43-90363aa1cfbd service cinder] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:33:48.172072 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e8f1cf-549a-4033-ab43-90363aa1cfbd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:33:48.181271 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e8f1cf-549a-4033-ab43-90363aa1cfbd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:33:48.182702 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e8f1cf-549a-4033-ab43-90363aa1cfbd service cinder] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:33:48.191987 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 760/1539] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Sep 6 04:33:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 04:33:57.567615 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=11 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:33:58.187846 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=33 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:35:13.037119 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d285ca-c975-4bd9-9786-7515b1c5226f None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:35:13.040478 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d285ca-c975-4bd9-9786-7515b1c5226f None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:35:13.040478 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d285ca-c975-4bd9-9786-7515b1c5226f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:35:13.063605 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-b4d285ca-c975-4bd9-9786-7515b1c5226f None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:35:13.065454 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-b4d285ca-c975-4bd9-9786-7515b1c5226f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:35:13.088712 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 780/1540] 10.222.0.30 () {58 vars in 958 bytes} [Tue Sep 6 04:35:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 04:35:13.098563 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a805c96-1830-475c-8d12-6d52c35fd770 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:35:13.099549 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a805c96-1830-475c-8d12-6d52c35fd770 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:35:13.100044 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a805c96-1830-475c-8d12-6d52c35fd770 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:35:13.126582 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-5a805c96-1830-475c-8d12-6d52c35fd770 None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:35:13.129537 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-5a805c96-1830-475c-8d12-6d52c35fd770 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:35:13.140089 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 761/1541] 10.222.0.30 () {58 vars in 958 bytes} [Tue Sep 6 04:35:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 04:35:13.144533 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d285ca-c975-4bd9-9786-7515b1c5226f None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:35:13.159433 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d285ca-c975-4bd9-9786-7515b1c5226f None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:35:13.164616 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32bac83-e37e-48d9-a976-08931bfd393c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:35:13.165601 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32bac83-e37e-48d9-a976-08931bfd393c admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:35:13.166115 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32bac83-e37e-48d9-a976-08931bfd393c admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:35:13.166598 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32bac83-e37e-48d9-a976-08931bfd393c admin admin] PATH_INFO: `/v3/domains` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:35:13.167637 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32bac83-e37e-48d9-a976-08931bfd393c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:35:13.168163 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32bac83-e37e-48d9-a976-08931bfd393c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:35:13.170018 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32bac83-e37e-48d9-a976-08931bfd393c admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:35:13.187452 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 781/1542] 10.222.0.30 () {58 vars in 1153 bytes} [Tue Sep 6 04:35:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:35:13.195735 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a805c96-1830-475c-8d12-6d52c35fd770 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:35:13.211395 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a805c96-1830-475c-8d12-6d52c35fd770 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:35:13.217597 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6be9a43-2154-4b51-9a7a-8bf4b27d7908 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:35:13.218422 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6be9a43-2154-4b51-9a7a-8bf4b27d7908 admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:35:13.221369 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6be9a43-2154-4b51-9a7a-8bf4b27d7908 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:35:13.221369 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6be9a43-2154-4b51-9a7a-8bf4b27d7908 admin admin] PATH_INFO: `/v3/projects` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:35:13.221369 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6be9a43-2154-4b51-9a7a-8bf4b27d7908 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:35:13.221848 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6be9a43-2154-4b51-9a7a-8bf4b27d7908 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:35:13.347375 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868c6f03-6c19-4565-95e5-871ad42ab61e None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:35:13.348336 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868c6f03-6c19-4565-95e5-871ad42ab61e None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:35:13.348873 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868c6f03-6c19-4565-95e5-871ad42ab61e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:35:13.364556 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 762/1543] 10.222.0.30 () {60 vars in 1153 bytes} [Tue Sep 6 04:35:13 2022] POST /identity/v3/projects => generated 367 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 04:35:13.371830 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6be9a43-2154-4b51-9a7a-8bf4b27d7908 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:35:13.402256 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-868c6f03-6c19-4565-95e5-871ad42ab61e None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:35:13.403718 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-868c6f03-6c19-4565-95e5-871ad42ab61e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:35:13.534005 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 782/1544] 10.222.0.30 () {58 vars in 958 bytes} [Tue Sep 6 04:35:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 04:35:13.538907 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d9e4b8-71ea-4a37-97ce-89b358a05a3a None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:35:13.539221 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d9e4b8-71ea-4a37-97ce-89b358a05a3a None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:35:13.539482 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d9e4b8-71ea-4a37-97ce-89b358a05a3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:35:13.550129 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6be9a43-2154-4b51-9a7a-8bf4b27d7908 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:35:13.567847 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-88d9e4b8-71ea-4a37-97ce-89b358a05a3a None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:35:13.569531 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-88d9e4b8-71ea-4a37-97ce-89b358a05a3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:35:13.571211 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d722f9-c79e-4db5-bbd4-85adee1e0145 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:35:13.572089 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d722f9-c79e-4db5-bbd4-85adee1e0145 admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:35:13.572656 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d722f9-c79e-4db5-bbd4-85adee1e0145 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:35:13.573229 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d722f9-c79e-4db5-bbd4-85adee1e0145 admin admin] PATH_INFO: `/v3/users` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:35:13.575028 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d722f9-c79e-4db5-bbd4-85adee1e0145 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:35:13.576341 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: WARNING py.warnings [None req-b7d722f9-c79e-4db5-bbd4-85adee1e0145 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 06 04:35:13.576341 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: warnings.warn(msg) Sep 06 04:35:13.576341 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: Sep 06 04:35:13.578168 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d722f9-c79e-4db5-bbd4-85adee1e0145 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:35:13.585334 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 783/1545] 10.222.0.30 () {58 vars in 958 bytes} [Tue Sep 6 04:35:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 04:35:13.585334 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958aedf3-2745-474e-8017-c06180b340b9 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:35:13.585334 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958aedf3-2745-474e-8017-c06180b340b9 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:35:13.585334 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958aedf3-2745-474e-8017-c06180b340b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:35:13.627288 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-958aedf3-2745-474e-8017-c06180b340b9 None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:35:13.627288 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-958aedf3-2745-474e-8017-c06180b340b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:35:13.634576 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 784/1546] 10.222.0.30 () {58 vars in 958 bytes} [Tue Sep 6 04:35:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 04:35:13.636499 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb8e936-33d3-4acd-aaec-96c2173ab88d None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:35:13.636914 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb8e936-33d3-4acd-aaec-96c2173ab88d None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:35:13.637412 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb8e936-33d3-4acd-aaec-96c2173ab88d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:35:13.665765 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-bfb8e936-33d3-4acd-aaec-96c2173ab88d None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:35:13.667252 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-bfb8e936-33d3-4acd-aaec-96c2173ab88d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:35:13.677056 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 785/1547] 10.222.0.30 () {58 vars in 958 bytes} [Tue Sep 6 04:35:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 04:35:13.678054 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb8e936-33d3-4acd-aaec-96c2173ab88d None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:35:13.690390 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb8e936-33d3-4acd-aaec-96c2173ab88d None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:35:13.696054 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d88cca3-3263-4e1e-85c0-d471187477d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:35:13.696868 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d88cca3-3263-4e1e-85c0-d471187477d6 admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:35:13.697156 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d88cca3-3263-4e1e-85c0-d471187477d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:35:13.697459 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d88cca3-3263-4e1e-85c0-d471187477d6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:35:13.698138 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d88cca3-3263-4e1e-85c0-d471187477d6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:35:13.698445 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d88cca3-3263-4e1e-85c0-d471187477d6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:35:13.700750 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d88cca3-3263-4e1e-85c0-d471187477d6 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:35:13.711462 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 763/1548] 10.222.0.30 () {60 vars in 1144 bytes} [Tue Sep 6 04:35:13 2022] POST /identity/v3/users => generated 284 bytes in 341 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 04:35:13.712630 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d722f9-c79e-4db5-bbd4-85adee1e0145 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:35:13.740313 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d722f9-c79e-4db5-bbd4-85adee1e0145 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:35:13.743678 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 786/1549] 10.222.0.30 () {58 vars in 1153 bytes} [Tue Sep 6 04:35:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:35:13.744543 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d88cca3-3263-4e1e-85c0-d471187477d6 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:35:13.754722 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f815d3-99d5-4c02-be04-3248ab0b7f06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:35:13.755341 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f815d3-99d5-4c02-be04-3248ab0b7f06 admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:35:13.755622 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f815d3-99d5-4c02-be04-3248ab0b7f06 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:35:13.755894 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f815d3-99d5-4c02-be04-3248ab0b7f06 admin admin] PATH_INFO: `/v3/domains` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:35:13.756552 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f815d3-99d5-4c02-be04-3248ab0b7f06 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:35:13.756858 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f815d3-99d5-4c02-be04-3248ab0b7f06 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:35:13.757893 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: WARNING py.warnings [None req-a0f815d3-99d5-4c02-be04-3248ab0b7f06 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 06 04:35:13.757893 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: warnings.warn(msg) Sep 06 04:35:13.757893 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: Sep 06 04:35:13.764857 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f815d3-99d5-4c02-be04-3248ab0b7f06 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:35:13.772805 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d88cca3-3263-4e1e-85c0-d471187477d6 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:35:13.783677 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991f5580-29cf-4907-be68-eb41d9f0f6db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:35:13.784370 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991f5580-29cf-4907-be68-eb41d9f0f6db admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:35:13.784663 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991f5580-29cf-4907-be68-eb41d9f0f6db admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:35:13.784936 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991f5580-29cf-4907-be68-eb41d9f0f6db admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:35:13.786209 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991f5580-29cf-4907-be68-eb41d9f0f6db admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:35:13.787861 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991f5580-29cf-4907-be68-eb41d9f0f6db admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:35:13.795703 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 764/1550] 10.222.0.30 () {58 vars in 1153 bytes} [Tue Sep 6 04:35:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:35:13.796556 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f815d3-99d5-4c02-be04-3248ab0b7f06 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:35:13.801427 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 787/1551] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Sep 6 04:35:13 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 04:35:13.805327 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991f5580-29cf-4907-be68-eb41d9f0f6db admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:35:13.818755 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991f5580-29cf-4907-be68-eb41d9f0f6db admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:35:13.823583 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f815d3-99d5-4c02-be04-3248ab0b7f06 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:35:13.825361 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3199e65c-1cc0-46ce-b053-bb6c9815fd53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:35:13.826088 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3199e65c-1cc0-46ce-b053-bb6c9815fd53 admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:35:13.826470 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3199e65c-1cc0-46ce-b053-bb6c9815fd53 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:35:13.826844 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3199e65c-1cc0-46ce-b053-bb6c9815fd53 admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:35:13.827588 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3199e65c-1cc0-46ce-b053-bb6c9815fd53 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:35:13.829325 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3199e65c-1cc0-46ce-b053-bb6c9815fd53 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:35:13.843493 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 788/1552] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Sep 6 04:35:13 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 06 04:35:13.844993 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3199e65c-1cc0-46ce-b053-bb6c9815fd53 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:35:13.850029 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8137b107-1a21-4dfe-86cc-227090dc23a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:35:13.850668 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8137b107-1a21-4dfe-86cc-227090dc23a2 admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:35:13.850952 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8137b107-1a21-4dfe-86cc-227090dc23a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:35:13.851229 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8137b107-1a21-4dfe-86cc-227090dc23a2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:35:13.851877 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8137b107-1a21-4dfe-86cc-227090dc23a2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:35:13.853926 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8137b107-1a21-4dfe-86cc-227090dc23a2 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:35:13.857239 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3199e65c-1cc0-46ce-b053-bb6c9815fd53 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:35:13.862556 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3094de0b-1ddf-4d79-9c2b-47a3d1b11323 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:35:13.863297 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3094de0b-1ddf-4d79-9c2b-47a3d1b11323 admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:35:13.863589 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3094de0b-1ddf-4d79-9c2b-47a3d1b11323 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:35:13.863867 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3094de0b-1ddf-4d79-9c2b-47a3d1b11323 admin admin] PATH_INFO: `/v3/projects` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:35:13.864842 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3094de0b-1ddf-4d79-9c2b-47a3d1b11323 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:35:13.866011 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: WARNING py.warnings [None req-3094de0b-1ddf-4d79-9c2b-47a3d1b11323 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 06 04:35:13.866011 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: warnings.warn(msg) Sep 06 04:35:13.866011 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: Sep 06 04:35:13.867490 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3094de0b-1ddf-4d79-9c2b-47a3d1b11323 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:35:13.993733 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 789/1553] 10.222.0.30 () {60 vars in 1153 bytes} [Tue Sep 6 04:35:13 2022] POST /identity/v3/projects => generated 377 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 04:35:13.994531 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3094de0b-1ddf-4d79-9c2b-47a3d1b11323 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:35:14.016019 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 765/1554] 10.222.0.30 () {60 vars in 1153 bytes} [Tue Sep 6 04:35:13 2022] POST /identity/v3/projects => generated 363 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 04:35:14.017459 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96902876-8c04-4ccf-8800-f7c20bac663a None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:35:14.017756 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96902876-8c04-4ccf-8800-f7c20bac663a None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:35:14.018051 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96902876-8c04-4ccf-8800-f7c20bac663a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:35:14.039083 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-96902876-8c04-4ccf-8800-f7c20bac663a None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:35:14.040311 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-96902876-8c04-4ccf-8800-f7c20bac663a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:35:14.057671 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3094de0b-1ddf-4d79-9c2b-47a3d1b11323 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:35:14.066030 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:35:14.099468 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 766/1555] 10.222.0.30 () {58 vars in 958 bytes} [Tue Sep 6 04:35:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 04:35:14.107872 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aee8f13-3f09-4b53-92b9-04a7944d1e21 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:35:14.107872 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aee8f13-3f09-4b53-92b9-04a7944d1e21 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:35:14.107872 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aee8f13-3f09-4b53-92b9-04a7944d1e21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:35:14.116967 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08bc195-3353-4855-9f25-984dee1d3057 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:35:14.117688 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08bc195-3353-4855-9f25-984dee1d3057 admin admin] REQUEST_METHOD: `PUT` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:35:14.117964 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08bc195-3353-4855-9f25-984dee1d3057 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:35:14.118260 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08bc195-3353-4855-9f25-984dee1d3057 admin admin] PATH_INFO: `/v3/projects/c0fa3c60782e4830b8c02ab8770e8d60/users/322f6308959a453f8178e405634a116e/roles/29607b2de39f40d68bb6ceef64b8a732` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:35:14.123273 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-8aee8f13-3f09-4b53-92b9-04a7944d1e21 None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:35:14.124551 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-8aee8f13-3f09-4b53-92b9-04a7944d1e21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:35:14.132509 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 767/1556] 10.222.0.30 () {58 vars in 958 bytes} [Tue Sep 6 04:35:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 04:35:14.133734 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa77b57-f795-49ac-b7e4-59e4615376cf None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:35:14.134023 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa77b57-f795-49ac-b7e4-59e4615376cf None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:35:14.134298 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa77b57-f795-49ac-b7e4-59e4615376cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:35:14.139680 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c08bc195-3353-4855-9f25-984dee1d3057 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c0fa3c60782e4830b8c02ab8770e8d60, user_id=322f6308959a453f8178e405634a116e, role_id=29607b2de39f40d68bb6ceef64b8a732)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:35:14.140861 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: WARNING py.warnings [None req-c08bc195-3353-4855-9f25-984dee1d3057 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 06 04:35:14.140861 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: warnings.warn(msg) Sep 06 04:35:14.140861 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: Sep 06 04:35:14.143940 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c08bc195-3353-4855-9f25-984dee1d3057 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:35:14.158828 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-6aa77b57-f795-49ac-b7e4-59e4615376cf None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:35:14.158828 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-6aa77b57-f795-49ac-b7e4-59e4615376cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:35:14.162563 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 768/1557] 10.222.0.30 () {58 vars in 958 bytes} [Tue Sep 6 04:35:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 04:35:14.163730 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec2e158-94db-4b53-8041-5843e9bc93da None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:35:14.164012 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec2e158-94db-4b53-8041-5843e9bc93da None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:35:14.164287 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec2e158-94db-4b53-8041-5843e9bc93da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:35:14.173220 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:35:14.184504 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-9ec2e158-94db-4b53-8041-5843e9bc93da None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:35:14.187600 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-9ec2e158-94db-4b53-8041-5843e9bc93da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:35:14.193619 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 769/1558] 10.222.0.30 () {58 vars in 958 bytes} [Tue Sep 6 04:35:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 04:35:14.194785 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7fc470-36be-4397-817e-2290cca71351 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:35:14.195061 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7fc470-36be-4397-817e-2290cca71351 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:35:14.195335 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7fc470-36be-4397-817e-2290cca71351 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:35:14.213013 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 790/1559] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Sep 6 04:35:13 2022] PUT /identity/v3/projects/c0fa3c60782e4830b8c02ab8770e8d60/users/322f6308959a453f8178e405634a116e/roles/29607b2de39f40d68bb6ceef64b8a732 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 04:35:14.213906 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-4f7fc470-36be-4397-817e-2290cca71351 None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:35:14.214385 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08bc195-3353-4855-9f25-984dee1d3057 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:35:14.215488 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-4f7fc470-36be-4397-817e-2290cca71351 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:35:14.275934 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 770/1560] 10.222.0.30 () {58 vars in 958 bytes} [Tue Sep 6 04:35:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 04:35:14.277145 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7fc470-36be-4397-817e-2290cca71351 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:35:14.283833 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08bc195-3353-4855-9f25-984dee1d3057 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:35:14.287066 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7fc470-36be-4397-817e-2290cca71351 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:35:14.288688 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a2986c-e3f6-493c-957c-fbf0acd72137 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:35:14.289333 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a2986c-e3f6-493c-957c-fbf0acd72137 admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:35:14.289648 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a2986c-e3f6-493c-957c-fbf0acd72137 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:35:14.289924 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a2986c-e3f6-493c-957c-fbf0acd72137 admin admin] PATH_INFO: `/v3/users` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:35:14.290559 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a2986c-e3f6-493c-957c-fbf0acd72137 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:35:14.293103 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a2986c-e3f6-493c-957c-fbf0acd72137 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:35:14.293474 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25786ca-1cbe-409d-97bd-d3df916fa684 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:35:14.294278 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25786ca-1cbe-409d-97bd-d3df916fa684 admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:35:14.294563 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25786ca-1cbe-409d-97bd-d3df916fa684 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:35:14.294839 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25786ca-1cbe-409d-97bd-d3df916fa684 admin admin] PATH_INFO: `/v3/users` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:35:14.295473 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25786ca-1cbe-409d-97bd-d3df916fa684 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:35:14.297510 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25786ca-1cbe-409d-97bd-d3df916fa684 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:35:14.381609 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 791/1561] 10.222.0.30 () {60 vars in 1144 bytes} [Tue Sep 6 04:35:14 2022] POST /identity/v3/users => generated 289 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 04:35:14.383370 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b8327a-25df-4ff9-a3f6-9d00840d25a0 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:35:14.383822 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b8327a-25df-4ff9-a3f6-9d00840d25a0 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:35:14.384543 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b8327a-25df-4ff9-a3f6-9d00840d25a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:35:14.407599 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 771/1562] 10.222.0.30 () {60 vars in 1144 bytes} [Tue Sep 6 04:35:14 2022] POST /identity/v3/users => generated 282 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 04:35:14.407599 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61defd0-1d1b-49bb-9883-fe6872c3f46f None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:35:14.407926 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-e4b8327a-25df-4ff9-a3f6-9d00840d25a0 None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:35:14.408033 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61defd0-1d1b-49bb-9883-fe6872c3f46f None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:35:14.408033 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61defd0-1d1b-49bb-9883-fe6872c3f46f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:35:14.408237 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-e4b8327a-25df-4ff9-a3f6-9d00840d25a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:35:14.416070 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 792/1563] 10.222.0.30 () {58 vars in 958 bytes} [Tue Sep 6 04:35:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 04:35:14.417288 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e518ff4-9390-4566-ab32-5bb8857bfc8c None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:35:14.418181 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e518ff4-9390-4566-ab32-5bb8857bfc8c None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:35:14.418507 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e518ff4-9390-4566-ab32-5bb8857bfc8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:35:14.426624 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-a61defd0-1d1b-49bb-9883-fe6872c3f46f None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:35:14.430338 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-a61defd0-1d1b-49bb-9883-fe6872c3f46f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:35:14.436141 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 772/1564] 10.222.0.30 () {58 vars in 958 bytes} [Tue Sep 6 04:35:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 04:35:14.437394 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357f26d8-2540-4fbf-aa8b-52d8942fb883 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:35:14.437674 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357f26d8-2540-4fbf-aa8b-52d8942fb883 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:35:14.437947 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357f26d8-2540-4fbf-aa8b-52d8942fb883 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:35:14.438557 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-5e518ff4-9390-4566-ab32-5bb8857bfc8c None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:35:14.440002 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-5e518ff4-9390-4566-ab32-5bb8857bfc8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:35:14.447859 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 793/1565] 10.222.0.30 () {58 vars in 958 bytes} [Tue Sep 6 04:35:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 04:35:14.449728 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443c7cb9-9bc8-40d9-af86-c412f8438580 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:35:14.449728 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443c7cb9-9bc8-40d9-af86-c412f8438580 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:35:14.450110 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443c7cb9-9bc8-40d9-af86-c412f8438580 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:35:14.458335 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-357f26d8-2540-4fbf-aa8b-52d8942fb883 None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:35:14.459543 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-357f26d8-2540-4fbf-aa8b-52d8942fb883 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:35:14.468187 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 773/1566] 10.222.0.30 () {58 vars in 958 bytes} [Tue Sep 6 04:35:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 04:35:14.470517 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35033b81-ca98-40ee-a86c-9a18318eac7f None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:35:14.470865 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35033b81-ca98-40ee-a86c-9a18318eac7f None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:35:14.471327 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35033b81-ca98-40ee-a86c-9a18318eac7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:35:14.479617 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-443c7cb9-9bc8-40d9-af86-c412f8438580 None None] MFA Rules not processed for user `322f6308959a453f8178e405634a116e`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:35:14.481710 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-443c7cb9-9bc8-40d9-af86-c412f8438580 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:35:14.491931 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-35033b81-ca98-40ee-a86c-9a18318eac7f None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:35:14.493186 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-35033b81-ca98-40ee-a86c-9a18318eac7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:35:14.501163 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 774/1567] 10.222.0.30 () {58 vars in 958 bytes} [Tue Sep 6 04:35:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 04:35:14.501981 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35033b81-ca98-40ee-a86c-9a18318eac7f None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:35:14.511596 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35033b81-ca98-40ee-a86c-9a18318eac7f None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:35:14.516576 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d97bfda-edf4-417b-b136-349d70afdbd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:35:14.517209 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d97bfda-edf4-417b-b136-349d70afdbd7 admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:35:14.517516 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d97bfda-edf4-417b-b136-349d70afdbd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:35:14.517789 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d97bfda-edf4-417b-b136-349d70afdbd7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:35:14.518428 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d97bfda-edf4-417b-b136-349d70afdbd7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:35:14.520027 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d97bfda-edf4-417b-b136-349d70afdbd7 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:35:14.531000 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 775/1568] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Sep 6 04:35:14 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 04:35:14.531865 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d97bfda-edf4-417b-b136-349d70afdbd7 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:35:14.541428 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d97bfda-edf4-417b-b136-349d70afdbd7 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:35:14.544484 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:35:14.546665 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5d6b78-20ab-4651-9afa-c5d62c0b5c17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:35:14.547260 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5d6b78-20ab-4651-9afa-c5d62c0b5c17 admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:35:14.547538 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5d6b78-20ab-4651-9afa-c5d62c0b5c17 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:35:14.547810 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5d6b78-20ab-4651-9afa-c5d62c0b5c17 admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:35:14.548345 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 794/1569] 10.222.0.30 () {58 vars in 958 bytes} [Tue Sep 6 04:35:14 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 04:35:14.548464 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5d6b78-20ab-4651-9afa-c5d62c0b5c17 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:35:14.549133 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443c7cb9-9bc8-40d9-af86-c412f8438580 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:35:14.550130 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5d6b78-20ab-4651-9afa-c5d62c0b5c17 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:35:14.561251 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443c7cb9-9bc8-40d9-af86-c412f8438580 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:35:14.561744 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 776/1570] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Sep 6 04:35:14 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 04:35:14.562597 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5d6b78-20ab-4651-9afa-c5d62c0b5c17 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:35:14.568420 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f378592-b3c3-421e-b6dd-cea75647280c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:35:14.569031 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f378592-b3c3-421e-b6dd-cea75647280c admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:35:14.569321 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f378592-b3c3-421e-b6dd-cea75647280c admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:35:14.569605 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f378592-b3c3-421e-b6dd-cea75647280c admin admin] PATH_INFO: `/v3/domains` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:35:14.570249 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f378592-b3c3-421e-b6dd-cea75647280c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:35:14.570546 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f378592-b3c3-421e-b6dd-cea75647280c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:35:14.572145 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f378592-b3c3-421e-b6dd-cea75647280c admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:35:14.574331 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5d6b78-20ab-4651-9afa-c5d62c0b5c17 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:35:14.580378 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2604846-322d-4fcc-9d25-acc0297111a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:35:14.581012 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2604846-322d-4fcc-9d25-acc0297111a5 admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:35:14.581414 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2604846-322d-4fcc-9d25-acc0297111a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:35:14.587558 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2604846-322d-4fcc-9d25-acc0297111a5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:35:14.587558 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2604846-322d-4fcc-9d25-acc0297111a5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:35:14.587558 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2604846-322d-4fcc-9d25-acc0297111a5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:35:14.587558 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2604846-322d-4fcc-9d25-acc0297111a5 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:35:14.587891 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 795/1571] 10.222.0.30 () {58 vars in 1153 bytes} [Tue Sep 6 04:35:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:35:14.587891 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f378592-b3c3-421e-b6dd-cea75647280c admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:35:14.596446 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f378592-b3c3-421e-b6dd-cea75647280c admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:35:14.603571 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 777/1572] 10.222.0.30 () {58 vars in 1153 bytes} [Tue Sep 6 04:35:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:35:14.603571 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2604846-322d-4fcc-9d25-acc0297111a5 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:35:14.607681 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a8876f-7249-4e0e-8a76-78ec78a62069 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:35:14.608307 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a8876f-7249-4e0e-8a76-78ec78a62069 admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:35:14.608585 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a8876f-7249-4e0e-8a76-78ec78a62069 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:35:14.608854 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a8876f-7249-4e0e-8a76-78ec78a62069 admin admin] PATH_INFO: `/v3/domains` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:35:14.609556 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a8876f-7249-4e0e-8a76-78ec78a62069 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:35:14.609812 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2604846-322d-4fcc-9d25-acc0297111a5 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:35:14.609923 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a8876f-7249-4e0e-8a76-78ec78a62069 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:35:14.611493 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a8876f-7249-4e0e-8a76-78ec78a62069 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:35:14.614486 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd01b4ff-41fd-473f-a225-89d0a581072b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:35:14.622085 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd01b4ff-41fd-473f-a225-89d0a581072b admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:35:14.622085 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd01b4ff-41fd-473f-a225-89d0a581072b admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:35:14.622085 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd01b4ff-41fd-473f-a225-89d0a581072b admin admin] PATH_INFO: `/v3/domains` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:35:14.622085 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd01b4ff-41fd-473f-a225-89d0a581072b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:35:14.622085 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd01b4ff-41fd-473f-a225-89d0a581072b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:35:14.622085 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd01b4ff-41fd-473f-a225-89d0a581072b admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:35:14.625060 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 796/1573] 10.222.0.30 () {58 vars in 1153 bytes} [Tue Sep 6 04:35:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:35:14.625903 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a8876f-7249-4e0e-8a76-78ec78a62069 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:35:14.635547 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a8876f-7249-4e0e-8a76-78ec78a62069 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:35:14.637084 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 778/1574] 10.222.0.30 () {58 vars in 1153 bytes} [Tue Sep 6 04:35:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:35:14.638341 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd01b4ff-41fd-473f-a225-89d0a581072b admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:35:14.641531 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ae3aef-e197-4940-8da1-e879e92063d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:35:14.642263 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ae3aef-e197-4940-8da1-e879e92063d6 admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:35:14.642551 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ae3aef-e197-4940-8da1-e879e92063d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:35:14.642823 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ae3aef-e197-4940-8da1-e879e92063d6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:35:14.643470 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ae3aef-e197-4940-8da1-e879e92063d6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:35:14.643775 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ae3aef-e197-4940-8da1-e879e92063d6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:35:14.645413 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ae3aef-e197-4940-8da1-e879e92063d6 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:35:14.649256 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd01b4ff-41fd-473f-a225-89d0a581072b admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:35:14.654211 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8648ef-5d63-48d0-b78a-9d5537568049 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:35:14.654815 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8648ef-5d63-48d0-b78a-9d5537568049 admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:35:14.655098 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8648ef-5d63-48d0-b78a-9d5537568049 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:35:14.655371 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8648ef-5d63-48d0-b78a-9d5537568049 admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:35:14.656055 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8648ef-5d63-48d0-b78a-9d5537568049 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:35:14.657684 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8648ef-5d63-48d0-b78a-9d5537568049 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:35:14.657929 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 797/1575] 10.222.0.30 () {58 vars in 1153 bytes} [Tue Sep 6 04:35:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:35:14.658731 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ae3aef-e197-4940-8da1-e879e92063d6 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:35:14.675055 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 779/1576] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Sep 6 04:35:14 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 04:35:14.675055 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8648ef-5d63-48d0-b78a-9d5537568049 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:35:14.682373 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8648ef-5d63-48d0-b78a-9d5537568049 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:35:14.687911 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d82c78b-acd5-4e66-b856-9309ad63b8ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:35:14.688504 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d82c78b-acd5-4e66-b856-9309ad63b8ab admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:35:14.688786 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d82c78b-acd5-4e66-b856-9309ad63b8ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:35:14.689059 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d82c78b-acd5-4e66-b856-9309ad63b8ab admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:35:14.689737 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d82c78b-acd5-4e66-b856-9309ad63b8ab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:35:14.691324 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d82c78b-acd5-4e66-b856-9309ad63b8ab admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:35:14.701546 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 780/1577] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Sep 6 04:35:14 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 04:35:14.702393 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d82c78b-acd5-4e66-b856-9309ad63b8ab admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:35:14.713521 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d82c78b-acd5-4e66-b856-9309ad63b8ab admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:35:14.715133 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ae3aef-e197-4940-8da1-e879e92063d6 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:35:14.718354 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5790b028-b5c3-48fe-9439-48873c05776d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:35:14.718826 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4cf3c36-d0c2-45c6-ba1c-d48b3088d11f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd09b5ab675634a2eb4b03755743f7d0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b707733ce65c42edbc1fd95ad0a1c06c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:35:14.718957 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5790b028-b5c3-48fe-9439-48873c05776d admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:35:14.719201 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5790b028-b5c3-48fe-9439-48873c05776d admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:35:14.719426 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cf3c36-d0c2-45c6-ba1c-d48b3088d11f service cinder] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:35:14.719525 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5790b028-b5c3-48fe-9439-48873c05776d admin admin] PATH_INFO: `/v3/projects` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:35:14.719726 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cf3c36-d0c2-45c6-ba1c-d48b3088d11f service cinder] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:35:14.719997 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cf3c36-d0c2-45c6-ba1c-d48b3088d11f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:35:14.720112 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5790b028-b5c3-48fe-9439-48873c05776d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:35:14.721991 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5790b028-b5c3-48fe-9439-48873c05776d admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:35:14.728432 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4cf3c36-d0c2-45c6-ba1c-d48b3088d11f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:35:14.730654 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4cf3c36-d0c2-45c6-ba1c-d48b3088d11f service cinder] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:35:14.740032 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 798/1578] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Sep 6 04:35:14 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 04:35:14.740811 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4cf3c36-d0c2-45c6-ba1c-d48b3088d11f service cinder] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:35:14.751385 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4cf3c36-d0c2-45c6-ba1c-d48b3088d11f service cinder] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:35:14.756055 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55602e86-f81d-47dc-973e-c52f8577d2a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:35:14.756713 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55602e86-f81d-47dc-973e-c52f8577d2a0 admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:35:14.757064 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55602e86-f81d-47dc-973e-c52f8577d2a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:35:14.757439 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55602e86-f81d-47dc-973e-c52f8577d2a0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:35:14.758139 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55602e86-f81d-47dc-973e-c52f8577d2a0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:35:14.760102 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55602e86-f81d-47dc-973e-c52f8577d2a0 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:35:14.827779 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 781/1579] 10.222.0.30 () {60 vars in 1153 bytes} [Tue Sep 6 04:35:14 2022] POST /identity/v3/projects => generated 393 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 04:35:14.828661 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5790b028-b5c3-48fe-9439-48873c05776d admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:35:14.861082 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:35:14.878180 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 799/1580] 10.222.0.30 () {60 vars in 1153 bytes} [Tue Sep 6 04:35:14 2022] POST /identity/v3/projects => generated 385 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 04:35:14.879200 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55602e86-f81d-47dc-973e-c52f8577d2a0 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:35:14.899812 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5790b028-b5c3-48fe-9439-48873c05776d admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:35:14.955237 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55602e86-f81d-47dc-973e-c52f8577d2a0 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:35:14.960081 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122fa8ff-7458-4f12-888d-5606181f082b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:35:14.960701 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122fa8ff-7458-4f12-888d-5606181f082b admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:35:14.961043 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122fa8ff-7458-4f12-888d-5606181f082b admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:35:14.961319 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122fa8ff-7458-4f12-888d-5606181f082b admin admin] PATH_INFO: `/v3/projects` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:35:14.962312 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122fa8ff-7458-4f12-888d-5606181f082b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:35:14.964236 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122fa8ff-7458-4f12-888d-5606181f082b admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:35:14.966770 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88626583-25d6-421b-82ca-642ef3c10649 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:35:14.967449 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88626583-25d6-421b-82ca-642ef3c10649 admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:35:14.967707 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88626583-25d6-421b-82ca-642ef3c10649 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:35:14.967981 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88626583-25d6-421b-82ca-642ef3c10649 admin admin] PATH_INFO: `/v3/projects` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:35:14.968659 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88626583-25d6-421b-82ca-642ef3c10649 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:35:14.970531 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88626583-25d6-421b-82ca-642ef3c10649 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:35:15.055719 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 800/1581] 10.222.0.30 () {60 vars in 1153 bytes} [Tue Sep 6 04:35:14 2022] POST /identity/v3/projects => generated 385 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 04:35:15.056549 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122fa8ff-7458-4f12-888d-5606181f082b admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:35:15.084859 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 782/1582] 10.222.0.30 () {60 vars in 1153 bytes} [Tue Sep 6 04:35:14 2022] POST /identity/v3/projects => generated 379 bytes in 257 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 04:35:15.085981 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88626583-25d6-421b-82ca-642ef3c10649 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:35:15.091742 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:35:15.124211 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122fa8ff-7458-4f12-888d-5606181f082b admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:35:15.152247 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88626583-25d6-421b-82ca-642ef3c10649 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:35:15.157456 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a313a68-4b40-40c0-a759-24d558927d43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:35:15.158241 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a313a68-4b40-40c0-a759-24d558927d43 admin admin] REQUEST_METHOD: `PUT` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:35:15.158591 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a313a68-4b40-40c0-a759-24d558927d43 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:35:15.158871 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a313a68-4b40-40c0-a759-24d558927d43 admin admin] PATH_INFO: `/v3/projects/a3ab4d7d42b2424a988365042e839456/users/fe78cd0f722440f2a42d71d2df164ebe/roles/29607b2de39f40d68bb6ceef64b8a732` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:35:15.173926 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a313a68-4b40-40c0-a759-24d558927d43 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a3ab4d7d42b2424a988365042e839456, user_id=fe78cd0f722440f2a42d71d2df164ebe, role_id=29607b2de39f40d68bb6ceef64b8a732)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:35:15.176568 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a313a68-4b40-40c0-a759-24d558927d43 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:35:15.180560 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0604817c-81c3-4e9a-9261-fd158e9ea9b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:35:15.181168 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0604817c-81c3-4e9a-9261-fd158e9ea9b3 admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:35:15.181554 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0604817c-81c3-4e9a-9261-fd158e9ea9b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:35:15.181830 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0604817c-81c3-4e9a-9261-fd158e9ea9b3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:35:15.182465 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0604817c-81c3-4e9a-9261-fd158e9ea9b3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:35:15.185157 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0604817c-81c3-4e9a-9261-fd158e9ea9b3 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:35:15.278422 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 783/1583] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Sep 6 04:35:15 2022] PUT /identity/v3/projects/a3ab4d7d42b2424a988365042e839456/users/fe78cd0f722440f2a42d71d2df164ebe/roles/29607b2de39f40d68bb6ceef64b8a732 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 04:35:15.279556 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a313a68-4b40-40c0-a759-24d558927d43 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:35:15.289342 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 801/1584] 10.222.0.30 () {60 vars in 1153 bytes} [Tue Sep 6 04:35:15 2022] POST /identity/v3/projects => generated 387 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 04:35:15.290204 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0604817c-81c3-4e9a-9261-fd158e9ea9b3 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:35:15.328640 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:35:15.346412 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a313a68-4b40-40c0-a759-24d558927d43 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:35:15.361714 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0604817c-81c3-4e9a-9261-fd158e9ea9b3 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:35:15.366633 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86195867-cdb2-494e-9aeb-d39774f6fe29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:35:15.367339 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86195867-cdb2-494e-9aeb-d39774f6fe29 admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:35:15.367675 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86195867-cdb2-494e-9aeb-d39774f6fe29 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:35:15.368029 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86195867-cdb2-494e-9aeb-d39774f6fe29 admin admin] PATH_INFO: `/v3/users` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:35:15.368768 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86195867-cdb2-494e-9aeb-d39774f6fe29 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:35:15.370791 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86195867-cdb2-494e-9aeb-d39774f6fe29 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:35:15.410262 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb2544f-3337-4202-90b0-c77a4ea01385 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:35:15.410928 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb2544f-3337-4202-90b0-c77a4ea01385 admin admin] REQUEST_METHOD: `PUT` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:35:15.411255 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb2544f-3337-4202-90b0-c77a4ea01385 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:35:15.411578 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb2544f-3337-4202-90b0-c77a4ea01385 admin admin] PATH_INFO: `/v3/projects/6852f495350f402a9b2280e91d2df088/users/5dcd5ff1bdc342539fb4fa154878bf3d/roles/29607b2de39f40d68bb6ceef64b8a732` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:35:15.425841 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb2544f-3337-4202-90b0-c77a4ea01385 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6852f495350f402a9b2280e91d2df088, user_id=5dcd5ff1bdc342539fb4fa154878bf3d, role_id=29607b2de39f40d68bb6ceef64b8a732)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:35:15.429554 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb2544f-3337-4202-90b0-c77a4ea01385 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:35:15.442910 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 802/1585] 10.222.0.30 () {60 vars in 1144 bytes} [Tue Sep 6 04:35:15 2022] POST /identity/v3/users => generated 297 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 04:35:15.443705 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86195867-cdb2-494e-9aeb-d39774f6fe29 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:35:15.454622 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86195867-cdb2-494e-9aeb-d39774f6fe29 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:35:15.459197 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b330816e-4d84-48c8-b50f-e062f4d6e420 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:35:15.459790 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b330816e-4d84-48c8-b50f-e062f4d6e420 admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:35:15.460065 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b330816e-4d84-48c8-b50f-e062f4d6e420 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:35:15.460335 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b330816e-4d84-48c8-b50f-e062f4d6e420 admin admin] PATH_INFO: `/v3/users` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:35:15.460960 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b330816e-4d84-48c8-b50f-e062f4d6e420 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:35:15.462905 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b330816e-4d84-48c8-b50f-e062f4d6e420 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:35:15.511572 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 784/1586] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Sep 6 04:35:15 2022] PUT /identity/v3/projects/6852f495350f402a9b2280e91d2df088/users/5dcd5ff1bdc342539fb4fa154878bf3d/roles/29607b2de39f40d68bb6ceef64b8a732 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 04:35:15.512432 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb2544f-3337-4202-90b0-c77a4ea01385 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:35:15.536770 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 803/1587] 10.222.0.30 () {60 vars in 1144 bytes} [Tue Sep 6 04:35:15 2022] POST /identity/v3/users => generated 293 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 04:35:15.538216 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b330816e-4d84-48c8-b50f-e062f4d6e420 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:35:15.556802 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b330816e-4d84-48c8-b50f-e062f4d6e420 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:35:15.591572 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb2544f-3337-4202-90b0-c77a4ea01385 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:35:15.596649 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04e71c3-1c7f-4db8-aa7a-1aa2eb904e26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:35:15.597280 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04e71c3-1c7f-4db8-aa7a-1aa2eb904e26 admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:35:15.597587 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04e71c3-1c7f-4db8-aa7a-1aa2eb904e26 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:35:15.597857 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04e71c3-1c7f-4db8-aa7a-1aa2eb904e26 admin admin] PATH_INFO: `/v3/users` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:35:15.598986 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c04e71c3-1c7f-4db8-aa7a-1aa2eb904e26 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:35:15.601281 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c04e71c3-1c7f-4db8-aa7a-1aa2eb904e26 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:35:15.616773 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:35:15.647819 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9720866a-71ca-4718-9b1d-51eff52214ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:35:15.648777 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9720866a-71ca-4718-9b1d-51eff52214ec admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:35:15.649216 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9720866a-71ca-4718-9b1d-51eff52214ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:35:15.649556 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9720866a-71ca-4718-9b1d-51eff52214ec admin admin] PATH_INFO: `/v3/users` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:35:15.650198 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9720866a-71ca-4718-9b1d-51eff52214ec admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:35:15.652285 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9720866a-71ca-4718-9b1d-51eff52214ec admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:35:15.678637 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 785/1588] 10.222.0.30 () {60 vars in 1144 bytes} [Tue Sep 6 04:35:15 2022] POST /identity/v3/users => generated 293 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 04:35:15.680242 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6ce961-f4fa-4f07-8bf0-d2d502ddc1e7 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:35:15.680742 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6ce961-f4fa-4f07-8bf0-d2d502ddc1e7 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:35:15.681236 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6ce961-f4fa-4f07-8bf0-d2d502ddc1e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:35:15.720101 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-9e6ce961-f4fa-4f07-8bf0-d2d502ddc1e7 None None] MFA Rules not processed for user `fe78cd0f722440f2a42d71d2df164ebe`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:35:15.721445 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-9e6ce961-f4fa-4f07-8bf0-d2d502ddc1e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:35:15.731973 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 804/1589] 10.222.0.30 () {60 vars in 1144 bytes} [Tue Sep 6 04:35:15 2022] POST /identity/v3/users => generated 290 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 04:35:15.731973 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9720866a-71ca-4718-9b1d-51eff52214ec admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:35:15.742355 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9720866a-71ca-4718-9b1d-51eff52214ec admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:35:15.748612 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0946e1d-169c-4b95-b75e-6b081b382fab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:35:15.754003 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0946e1d-169c-4b95-b75e-6b081b382fab admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:35:15.754003 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0946e1d-169c-4b95-b75e-6b081b382fab admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:35:15.754003 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0946e1d-169c-4b95-b75e-6b081b382fab admin admin] PATH_INFO: `/v3/users` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:35:15.754003 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0946e1d-169c-4b95-b75e-6b081b382fab admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:35:15.754003 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0946e1d-169c-4b95-b75e-6b081b382fab admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:35:15.797333 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:35:15.807485 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 786/1590] 10.222.0.30 () {58 vars in 958 bytes} [Tue Sep 6 04:35:15 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 04:35:15.810314 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6ce961-f4fa-4f07-8bf0-d2d502ddc1e7 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:35:15.821056 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6ce961-f4fa-4f07-8bf0-d2d502ddc1e7 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:35:15.826279 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622daa54-7f65-44b6-8cf7-9ffd7b5e3d16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:35:15.826891 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622daa54-7f65-44b6-8cf7-9ffd7b5e3d16 admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:35:15.827175 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622daa54-7f65-44b6-8cf7-9ffd7b5e3d16 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:35:15.827279 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 805/1591] 10.222.0.30 () {60 vars in 1144 bytes} [Tue Sep 6 04:35:15 2022] POST /identity/v3/users => generated 294 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 04:35:15.828601 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b386e3-66b8-445c-aa31-677c62a586e7 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:35:15.828920 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b386e3-66b8-445c-aa31-677c62a586e7 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:35:15.829195 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b386e3-66b8-445c-aa31-677c62a586e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:35:15.829553 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622daa54-7f65-44b6-8cf7-9ffd7b5e3d16 admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:35:15.830530 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622daa54-7f65-44b6-8cf7-9ffd7b5e3d16 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:35:15.832203 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622daa54-7f65-44b6-8cf7-9ffd7b5e3d16 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:35:15.843681 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 787/1592] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Sep 6 04:35:15 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 04:35:15.844564 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622daa54-7f65-44b6-8cf7-9ffd7b5e3d16 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:35:15.855000 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622daa54-7f65-44b6-8cf7-9ffd7b5e3d16 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:35:15.859771 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f72b71-b9c6-4a45-b372-12831ba955a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:35:15.859920 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-a8b386e3-66b8-445c-aa31-677c62a586e7 None None] MFA Rules not processed for user `5dcd5ff1bdc342539fb4fa154878bf3d`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:35:15.860376 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f72b71-b9c6-4a45-b372-12831ba955a9 admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:35:15.860664 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f72b71-b9c6-4a45-b372-12831ba955a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:35:15.860939 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f72b71-b9c6-4a45-b372-12831ba955a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:35:15.861152 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-a8b386e3-66b8-445c-aa31-677c62a586e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:35:15.861609 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f72b71-b9c6-4a45-b372-12831ba955a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:35:15.863201 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f72b71-b9c6-4a45-b372-12831ba955a9 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:35:15.873918 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 788/1593] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Sep 6 04:35:15 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 04:35:15.874817 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f72b71-b9c6-4a45-b372-12831ba955a9 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:35:15.884199 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f72b71-b9c6-4a45-b372-12831ba955a9 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:35:15.889810 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02fde057-8b53-486e-95f0-1aeb889da4d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:35:15.890418 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fde057-8b53-486e-95f0-1aeb889da4d5 admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:35:15.890701 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fde057-8b53-486e-95f0-1aeb889da4d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:35:15.891053 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fde057-8b53-486e-95f0-1aeb889da4d5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:35:15.891653 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02fde057-8b53-486e-95f0-1aeb889da4d5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:35:15.893902 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02fde057-8b53-486e-95f0-1aeb889da4d5 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:35:15.906784 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 789/1594] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Sep 6 04:35:15 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 04:35:15.907616 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02fde057-8b53-486e-95f0-1aeb889da4d5 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:35:15.918177 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02fde057-8b53-486e-95f0-1aeb889da4d5 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:35:15.923184 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70f32e2-b761-42a7-90a6-3cf5f474f551 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:35:15.923779 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70f32e2-b761-42a7-90a6-3cf5f474f551 admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:35:15.924054 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70f32e2-b761-42a7-90a6-3cf5f474f551 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:35:15.924325 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70f32e2-b761-42a7-90a6-3cf5f474f551 admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:35:15.925097 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70f32e2-b761-42a7-90a6-3cf5f474f551 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:35:15.926827 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70f32e2-b761-42a7-90a6-3cf5f474f551 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:35:15.936392 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 806/1595] 10.222.0.30 () {58 vars in 958 bytes} [Tue Sep 6 04:35:15 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 04:35:15.939950 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 790/1596] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Sep 6 04:35:15 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 04:35:15.940076 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b386e3-66b8-445c-aa31-677c62a586e7 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:35:15.940182 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70f32e2-b761-42a7-90a6-3cf5f474f551 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:35:15.948678 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70f32e2-b761-42a7-90a6-3cf5f474f551 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:35:15.950124 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b386e3-66b8-445c-aa31-677c62a586e7 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:35:15.954308 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07db83dd-04e5-42e7-b2ff-ac8841cd2c92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:35:15.962622 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07db83dd-04e5-42e7-b2ff-ac8841cd2c92 admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:35:15.962755 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9871b95-0562-4ef3-9256-9ac7a4021528 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:35:15.962880 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07db83dd-04e5-42e7-b2ff-ac8841cd2c92 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:35:15.962880 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07db83dd-04e5-42e7-b2ff-ac8841cd2c92 admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:35:15.963065 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9871b95-0562-4ef3-9256-9ac7a4021528 admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:35:15.963065 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9871b95-0562-4ef3-9256-9ac7a4021528 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:35:15.963247 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07db83dd-04e5-42e7-b2ff-ac8841cd2c92 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:35:15.963332 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9871b95-0562-4ef3-9256-9ac7a4021528 admin admin] PATH_INFO: `/v3/projects` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:35:15.963332 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9871b95-0562-4ef3-9256-9ac7a4021528 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:35:15.963503 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07db83dd-04e5-42e7-b2ff-ac8841cd2c92 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:35:15.963622 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9871b95-0562-4ef3-9256-9ac7a4021528 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:35:15.969645 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 791/1597] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Sep 6 04:35:15 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 04:35:15.970453 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07db83dd-04e5-42e7-b2ff-ac8841cd2c92 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:35:15.980019 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07db83dd-04e5-42e7-b2ff-ac8841cd2c92 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:35:15.984939 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa823f4c-0321-4747-a4a8-5e27f109e592 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:35:15.985561 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa823f4c-0321-4747-a4a8-5e27f109e592 admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:35:15.987987 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa823f4c-0321-4747-a4a8-5e27f109e592 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:35:15.987987 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa823f4c-0321-4747-a4a8-5e27f109e592 admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:35:15.987987 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa823f4c-0321-4747-a4a8-5e27f109e592 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:35:15.988382 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa823f4c-0321-4747-a4a8-5e27f109e592 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:35:15.999247 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 792/1598] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Sep 6 04:35:15 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 04:35:16.000017 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa823f4c-0321-4747-a4a8-5e27f109e592 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:35:16.009084 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa823f4c-0321-4747-a4a8-5e27f109e592 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:35:16.014598 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e446e56d-5992-4e34-988a-2e2f541229f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:35:16.015159 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e446e56d-5992-4e34-988a-2e2f541229f2 admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:35:16.015436 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e446e56d-5992-4e34-988a-2e2f541229f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:35:16.015707 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e446e56d-5992-4e34-988a-2e2f541229f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:35:16.016363 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e446e56d-5992-4e34-988a-2e2f541229f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:35:16.017941 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e446e56d-5992-4e34-988a-2e2f541229f2 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:35:16.028031 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 793/1599] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Sep 6 04:35:15 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 04:35:16.028789 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e446e56d-5992-4e34-988a-2e2f541229f2 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:35:16.038029 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e446e56d-5992-4e34-988a-2e2f541229f2 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:35:16.042924 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b530fe1-c09f-488d-843a-9ab7231c7950 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:35:16.043680 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b530fe1-c09f-488d-843a-9ab7231c7950 admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:35:16.043959 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b530fe1-c09f-488d-843a-9ab7231c7950 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:35:16.044232 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b530fe1-c09f-488d-843a-9ab7231c7950 admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:35:16.044851 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b530fe1-c09f-488d-843a-9ab7231c7950 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:35:16.046450 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b530fe1-c09f-488d-843a-9ab7231c7950 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:35:16.063594 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 807/1600] 10.222.0.30 () {60 vars in 1153 bytes} [Tue Sep 6 04:35:15 2022] POST /identity/v3/projects => generated 381 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 04:35:16.063594 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9871b95-0562-4ef3-9256-9ac7a4021528 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:35:16.063904 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 794/1601] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Sep 6 04:35:16 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 04:35:16.063904 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b530fe1-c09f-488d-843a-9ab7231c7950 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:35:16.074362 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b530fe1-c09f-488d-843a-9ab7231c7950 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:35:16.133188 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9871b95-0562-4ef3-9256-9ac7a4021528 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:35:16.133776 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8643da90-ee54-43dd-a7f8-3127ff01f204 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:35:16.134217 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8643da90-ee54-43dd-a7f8-3127ff01f204 admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:35:16.134494 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8643da90-ee54-43dd-a7f8-3127ff01f204 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:35:16.134764 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8643da90-ee54-43dd-a7f8-3127ff01f204 admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:35:16.135394 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8643da90-ee54-43dd-a7f8-3127ff01f204 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:35:16.137855 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8643da90-ee54-43dd-a7f8-3127ff01f204 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:35:16.138804 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e9e0a3-fd4f-4dea-99ce-e4f20ab211fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:35:16.139415 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e9e0a3-fd4f-4dea-99ce-e4f20ab211fa admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:35:16.139695 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e9e0a3-fd4f-4dea-99ce-e4f20ab211fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:35:16.139973 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e9e0a3-fd4f-4dea-99ce-e4f20ab211fa admin admin] PATH_INFO: `/v3/projects` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:35:16.140607 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e9e0a3-fd4f-4dea-99ce-e4f20ab211fa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:35:16.142532 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e9e0a3-fd4f-4dea-99ce-e4f20ab211fa admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:35:16.147827 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 795/1602] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Sep 6 04:35:16 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 04:35:16.149032 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8643da90-ee54-43dd-a7f8-3127ff01f204 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:35:16.158347 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8643da90-ee54-43dd-a7f8-3127ff01f204 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:35:16.162276 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:35:16.162905 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fefa3aa-3868-4b86-8d50-2c7e38abaf6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:35:16.163499 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fefa3aa-3868-4b86-8d50-2c7e38abaf6e admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:35:16.163774 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fefa3aa-3868-4b86-8d50-2c7e38abaf6e admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:35:16.164045 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fefa3aa-3868-4b86-8d50-2c7e38abaf6e admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:35:16.164668 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fefa3aa-3868-4b86-8d50-2c7e38abaf6e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:35:16.166274 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fefa3aa-3868-4b86-8d50-2c7e38abaf6e admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:35:16.178012 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 796/1603] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Sep 6 04:35:16 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 04:35:16.178917 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fefa3aa-3868-4b86-8d50-2c7e38abaf6e admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:35:16.188810 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fefa3aa-3868-4b86-8d50-2c7e38abaf6e admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:35:16.193214 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a26272-caf9-4a39-9378-343b86f81402 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:35:16.193895 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a26272-caf9-4a39-9378-343b86f81402 admin admin] REQUEST_METHOD: `PUT` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:35:16.194173 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a26272-caf9-4a39-9378-343b86f81402 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:35:16.194443 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a26272-caf9-4a39-9378-343b86f81402 admin admin] PATH_INFO: `/v3/projects/7aa4ca550edd4876a4c6c56deedf9d70/users/dec1c17550b849dbadbc16e98530b455/roles/29607b2de39f40d68bb6ceef64b8a732` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:35:16.209453 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a26272-caf9-4a39-9378-343b86f81402 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7aa4ca550edd4876a4c6c56deedf9d70, user_id=dec1c17550b849dbadbc16e98530b455, role_id=29607b2de39f40d68bb6ceef64b8a732)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:35:16.212050 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a26272-caf9-4a39-9378-343b86f81402 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:35:16.225606 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 808/1604] 10.222.0.30 () {60 vars in 1153 bytes} [Tue Sep 6 04:35:16 2022] POST /identity/v3/projects => generated 363 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 04:35:16.226417 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e9e0a3-fd4f-4dea-99ce-e4f20ab211fa admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:35:16.289918 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e9e0a3-fd4f-4dea-99ce-e4f20ab211fa admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:35:16.295335 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 797/1605] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Sep 6 04:35:16 2022] PUT /identity/v3/projects/7aa4ca550edd4876a4c6c56deedf9d70/users/dec1c17550b849dbadbc16e98530b455/roles/29607b2de39f40d68bb6ceef64b8a732 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 04:35:16.296072 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a26272-caf9-4a39-9378-343b86f81402 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:35:16.326628 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:35:16.349008 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8417b4ae-de6b-4f27-a1b2-b150ed8ddb21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:35:16.354542 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8417b4ae-de6b-4f27-a1b2-b150ed8ddb21 admin admin] REQUEST_METHOD: `PUT` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:35:16.354542 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8417b4ae-de6b-4f27-a1b2-b150ed8ddb21 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:35:16.354542 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8417b4ae-de6b-4f27-a1b2-b150ed8ddb21 admin admin] PATH_INFO: `/v3/projects/3d04ba57221f42d684873d624dbfd283/users/5a0294ea40914bf290902307a787642d/roles/29607b2de39f40d68bb6ceef64b8a732` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:35:16.359852 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a26272-caf9-4a39-9378-343b86f81402 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:35:16.365363 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7743ea8d-74da-4c4e-a251-f41600e9eabd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:35:16.366084 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7743ea8d-74da-4c4e-a251-f41600e9eabd admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:35:16.366440 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7743ea8d-74da-4c4e-a251-f41600e9eabd admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:35:16.366790 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7743ea8d-74da-4c4e-a251-f41600e9eabd admin admin] PATH_INFO: `/v3/users` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:35:16.367511 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7743ea8d-74da-4c4e-a251-f41600e9eabd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:35:16.372173 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7743ea8d-74da-4c4e-a251-f41600e9eabd admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:35:16.372324 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8417b4ae-de6b-4f27-a1b2-b150ed8ddb21 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3d04ba57221f42d684873d624dbfd283, user_id=5a0294ea40914bf290902307a787642d, role_id=29607b2de39f40d68bb6ceef64b8a732)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:35:16.374163 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8417b4ae-de6b-4f27-a1b2-b150ed8ddb21 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:35:16.458280 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 809/1606] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Sep 6 04:35:16 2022] PUT /identity/v3/projects/3d04ba57221f42d684873d624dbfd283/users/5a0294ea40914bf290902307a787642d/roles/29607b2de39f40d68bb6ceef64b8a732 => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 04:35:16.459089 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8417b4ae-de6b-4f27-a1b2-b150ed8ddb21 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:35:16.490237 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 798/1607] 10.222.0.30 () {60 vars in 1144 bytes} [Tue Sep 6 04:35:16 2022] POST /identity/v3/users => generated 297 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 04:35:16.490237 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7743ea8d-74da-4c4e-a251-f41600e9eabd admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:35:16.501175 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7743ea8d-74da-4c4e-a251-f41600e9eabd admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:35:16.524380 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8417b4ae-de6b-4f27-a1b2-b150ed8ddb21 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:35:16.529008 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655447f6-a86a-4b30-b35d-3d3d33146327 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:35:16.529687 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655447f6-a86a-4b30-b35d-3d3d33146327 admin admin] REQUEST_METHOD: `PUT` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:35:16.529970 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655447f6-a86a-4b30-b35d-3d3d33146327 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:35:16.530234 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655447f6-a86a-4b30-b35d-3d3d33146327 admin admin] PATH_INFO: `/v3/projects/e419b4601a80491d83a39dbf190f75c1/users/fca80ba4214e483691acda7540a33f02/roles/29607b2de39f40d68bb6ceef64b8a732` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:35:16.545470 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-655447f6-a86a-4b30-b35d-3d3d33146327 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e419b4601a80491d83a39dbf190f75c1, user_id=fca80ba4214e483691acda7540a33f02, role_id=29607b2de39f40d68bb6ceef64b8a732)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:35:16.548106 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-655447f6-a86a-4b30-b35d-3d3d33146327 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:35:16.561418 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d0137d-cf0b-4cd7-97e8-0574fa9d1163 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:35:16.562108 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d0137d-cf0b-4cd7-97e8-0574fa9d1163 admin admin] REQUEST_METHOD: `PUT` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:35:16.562405 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d0137d-cf0b-4cd7-97e8-0574fa9d1163 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:35:16.562676 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d0137d-cf0b-4cd7-97e8-0574fa9d1163 admin admin] PATH_INFO: `/v3/projects/96ba8ad37f17481a9691031fefbc3c80/users/15eebbd8b1d44f4897acb266633998db/roles/29607b2de39f40d68bb6ceef64b8a732` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:35:16.576782 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d0137d-cf0b-4cd7-97e8-0574fa9d1163 admin admin] RBAC: Authorizing `identity:create_grant(project_id=96ba8ad37f17481a9691031fefbc3c80, user_id=15eebbd8b1d44f4897acb266633998db, role_id=29607b2de39f40d68bb6ceef64b8a732)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:35:16.579502 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d0137d-cf0b-4cd7-97e8-0574fa9d1163 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:35:16.614471 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 810/1608] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Sep 6 04:35:16 2022] PUT /identity/v3/projects/e419b4601a80491d83a39dbf190f75c1/users/fca80ba4214e483691acda7540a33f02/roles/29607b2de39f40d68bb6ceef64b8a732 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 04:35:16.615286 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655447f6-a86a-4b30-b35d-3d3d33146327 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:35:16.625396 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:35:16.669247 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 799/1609] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Sep 6 04:35:16 2022] PUT /identity/v3/projects/96ba8ad37f17481a9691031fefbc3c80/users/15eebbd8b1d44f4897acb266633998db/roles/29607b2de39f40d68bb6ceef64b8a732 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 04:35:16.670252 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d0137d-cf0b-4cd7-97e8-0574fa9d1163 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:35:16.681568 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655447f6-a86a-4b30-b35d-3d3d33146327 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:35:16.738707 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d0137d-cf0b-4cd7-97e8-0574fa9d1163 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:35:16.739690 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a82efcd-5863-4089-a92e-e9dde12ee890 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:35:16.740361 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a82efcd-5863-4089-a92e-e9dde12ee890 admin admin] REQUEST_METHOD: `PUT` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:35:16.740670 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a82efcd-5863-4089-a92e-e9dde12ee890 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:35:16.740964 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a82efcd-5863-4089-a92e-e9dde12ee890 admin admin] PATH_INFO: `/v3/projects/2388589a34d346f3b31b231a9dafe34d/users/1ca147cfc97149dd87e533123a262cfd/roles/29607b2de39f40d68bb6ceef64b8a732` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:35:16.743926 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7218940-28e4-42f7-93d5-c576738f6ac1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:35:16.744539 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7218940-28e4-42f7-93d5-c576738f6ac1 admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:35:16.744818 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7218940-28e4-42f7-93d5-c576738f6ac1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:35:16.745092 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7218940-28e4-42f7-93d5-c576738f6ac1 admin admin] PATH_INFO: `/v3/users` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:35:16.745797 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7218940-28e4-42f7-93d5-c576738f6ac1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:35:16.747778 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7218940-28e4-42f7-93d5-c576738f6ac1 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:35:16.759303 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a82efcd-5863-4089-a92e-e9dde12ee890 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2388589a34d346f3b31b231a9dafe34d, user_id=1ca147cfc97149dd87e533123a262cfd, role_id=29607b2de39f40d68bb6ceef64b8a732)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:35:16.771684 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a82efcd-5863-4089-a92e-e9dde12ee890 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:35:16.852496 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 800/1610] 10.222.0.30 () {60 vars in 1144 bytes} [Tue Sep 6 04:35:16 2022] POST /identity/v3/users => generated 288 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 04:35:16.853704 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05639a7-8467-4240-8bb7-2bfb485a227c None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:35:16.853993 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05639a7-8467-4240-8bb7-2bfb485a227c None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:35:16.854298 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05639a7-8467-4240-8bb7-2bfb485a227c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:35:16.873192 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-e05639a7-8467-4240-8bb7-2bfb485a227c None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:35:16.874425 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-e05639a7-8467-4240-8bb7-2bfb485a227c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:35:16.882028 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 801/1611] 10.222.0.30 () {58 vars in 958 bytes} [Tue Sep 6 04:35:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 04:35:16.883194 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8128be-fb91-43ae-89d2-0b9cf220ff2d None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:35:16.887114 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8128be-fb91-43ae-89d2-0b9cf220ff2d None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:35:16.887754 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 811/1612] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Sep 6 04:35:16 2022] PUT /identity/v3/projects/2388589a34d346f3b31b231a9dafe34d/users/1ca147cfc97149dd87e533123a262cfd/roles/29607b2de39f40d68bb6ceef64b8a732 => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 04:35:16.887843 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8128be-fb91-43ae-89d2-0b9cf220ff2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:35:16.887918 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a82efcd-5863-4089-a92e-e9dde12ee890 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:35:16.903640 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-4e8128be-fb91-43ae-89d2-0b9cf220ff2d None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:35:16.905409 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-4e8128be-fb91-43ae-89d2-0b9cf220ff2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:35:16.923537 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:35:16.948318 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a82efcd-5863-4089-a92e-e9dde12ee890 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:35:16.954002 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6681530e-1554-4d81-972e-50159e38b790 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:35:16.954615 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6681530e-1554-4d81-972e-50159e38b790 admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:35:16.955009 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6681530e-1554-4d81-972e-50159e38b790 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:35:16.955297 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6681530e-1554-4d81-972e-50159e38b790 admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:35:16.955941 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6681530e-1554-4d81-972e-50159e38b790 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:35:16.957584 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6681530e-1554-4d81-972e-50159e38b790 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:35:16.966024 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 802/1613] 10.222.0.30 () {58 vars in 958 bytes} [Tue Sep 6 04:35:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 04:35:16.967517 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60b916b-c44b-4023-976e-fc6999aeb6ec None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:35:16.967801 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60b916b-c44b-4023-976e-fc6999aeb6ec None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:35:16.968074 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60b916b-c44b-4023-976e-fc6999aeb6ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:35:16.968802 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 812/1614] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Sep 6 04:35:16 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 04:35:16.970404 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c700124-1e05-4056-a91b-b870f76bae3f None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:35:16.970702 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c700124-1e05-4056-a91b-b870f76bae3f None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:35:16.970983 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c700124-1e05-4056-a91b-b870f76bae3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:35:16.990526 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-7c700124-1e05-4056-a91b-b870f76bae3f None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:35:16.991743 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-7c700124-1e05-4056-a91b-b870f76bae3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:35:16.998217 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-f60b916b-c44b-4023-976e-fc6999aeb6ec None None] MFA Rules not processed for user `fca80ba4214e483691acda7540a33f02`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:35:16.999396 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-f60b916b-c44b-4023-976e-fc6999aeb6ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:35:17.000648 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 813/1615] 10.222.0.30 () {58 vars in 958 bytes} [Tue Sep 6 04:35:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 04:35:17.001709 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c700124-1e05-4056-a91b-b870f76bae3f None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:35:17.011289 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c700124-1e05-4056-a91b-b870f76bae3f None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:35:17.016075 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2536633d-f47a-402b-ac5c-ad285c945573 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:35:17.016677 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2536633d-f47a-402b-ac5c-ad285c945573 admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:35:17.016949 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2536633d-f47a-402b-ac5c-ad285c945573 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:35:17.017219 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2536633d-f47a-402b-ac5c-ad285c945573 admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:35:17.017884 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2536633d-f47a-402b-ac5c-ad285c945573 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:35:17.019478 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2536633d-f47a-402b-ac5c-ad285c945573 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:35:17.033135 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 814/1616] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Sep 6 04:35:17 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 04:35:17.033135 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2f10bb-e7dd-453d-8529-ae2e0b5a7e8d None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:35:17.033135 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2f10bb-e7dd-453d-8529-ae2e0b5a7e8d None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:35:17.033135 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2f10bb-e7dd-453d-8529-ae2e0b5a7e8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:35:17.051305 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-6c2f10bb-e7dd-453d-8529-ae2e0b5a7e8d None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:35:17.052506 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-6c2f10bb-e7dd-453d-8529-ae2e0b5a7e8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:35:17.063932 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 815/1617] 10.222.0.30 () {58 vars in 958 bytes} [Tue Sep 6 04:35:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 04:35:17.063932 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2f10bb-e7dd-453d-8529-ae2e0b5a7e8d None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:35:17.071631 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 803/1618] 10.222.0.30 () {58 vars in 958 bytes} [Tue Sep 6 04:35:16 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 04:35:17.074056 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:35:17.075681 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60b916b-c44b-4023-976e-fc6999aeb6ec None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:35:17.087162 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60b916b-c44b-4023-976e-fc6999aeb6ec None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:35:17.091998 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b92bef-79bb-451f-ba1f-4ab89016bb10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:35:17.092654 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b92bef-79bb-451f-ba1f-4ab89016bb10 admin admin] REQUEST_METHOD: `PUT` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:35:17.092931 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b92bef-79bb-451f-ba1f-4ab89016bb10 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:35:17.093200 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b92bef-79bb-451f-ba1f-4ab89016bb10 admin admin] PATH_INFO: `/v3/projects/b4f8538b00954b3dacfd9ff9bead0e41/users/9df0ff1eb10946e192622a26876d1674/roles/ea4c814713b149108e488de855b025e4` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:35:17.108583 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b92bef-79bb-451f-ba1f-4ab89016bb10 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b4f8538b00954b3dacfd9ff9bead0e41, user_id=9df0ff1eb10946e192622a26876d1674, role_id=ea4c814713b149108e488de855b025e4)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:35:17.111214 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b92bef-79bb-451f-ba1f-4ab89016bb10 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:35:17.115939 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2f10bb-e7dd-453d-8529-ae2e0b5a7e8d None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:35:17.119592 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439a4004-22b7-4c55-9689-f02182c21c13 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90492440f6a6471e8392361c098428aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b707733ce65c42edbc1fd95ad0a1c06c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:35:17.120194 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439a4004-22b7-4c55-9689-f02182c21c13 service neutron] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:35:17.120552 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439a4004-22b7-4c55-9689-f02182c21c13 service neutron] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:35:17.120828 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439a4004-22b7-4c55-9689-f02182c21c13 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:35:17.130393 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-439a4004-22b7-4c55-9689-f02182c21c13 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:35:17.131819 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-439a4004-22b7-4c55-9689-f02182c21c13 service neutron] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:35:17.141813 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 816/1619] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Sep 6 04:35:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 04:35:17.142603 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439a4004-22b7-4c55-9689-f02182c21c13 service neutron] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:35:17.150513 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439a4004-22b7-4c55-9689-f02182c21c13 service neutron] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:35:17.155246 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504f5eb3-e007-44a2-9ce5-7f82d47632b4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90492440f6a6471e8392361c098428aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b707733ce65c42edbc1fd95ad0a1c06c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:35:17.155459 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504f5eb3-e007-44a2-9ce5-7f82d47632b4 service neutron] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:35:17.155750 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504f5eb3-e007-44a2-9ce5-7f82d47632b4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:35:17.156034 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504f5eb3-e007-44a2-9ce5-7f82d47632b4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:35:17.165071 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504f5eb3-e007-44a2-9ce5-7f82d47632b4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:35:17.166526 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504f5eb3-e007-44a2-9ce5-7f82d47632b4 service neutron] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:35:17.177466 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 817/1620] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Sep 6 04:35:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 04:35:17.178166 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504f5eb3-e007-44a2-9ce5-7f82d47632b4 service neutron] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:35:17.186931 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504f5eb3-e007-44a2-9ce5-7f82d47632b4 service neutron] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:35:17.187088 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 804/1621] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Sep 6 04:35:17 2022] PUT /identity/v3/projects/b4f8538b00954b3dacfd9ff9bead0e41/users/9df0ff1eb10946e192622a26876d1674/roles/ea4c814713b149108e488de855b025e4 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 04:35:17.187989 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b92bef-79bb-451f-ba1f-4ab89016bb10 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:35:17.246732 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b734b4-9bda-4d4a-9af3-bc60ea9a4a86 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90492440f6a6471e8392361c098428aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b707733ce65c42edbc1fd95ad0a1c06c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:35:17.247638 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b734b4-9bda-4d4a-9af3-bc60ea9a4a86 service neutron] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:35:17.247908 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b734b4-9bda-4d4a-9af3-bc60ea9a4a86 service neutron] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:35:17.248177 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b734b4-9bda-4d4a-9af3-bc60ea9a4a86 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:35:17.260684 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b92bef-79bb-451f-ba1f-4ab89016bb10 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:35:17.271321 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd76aa0-c191-4ff4-bef3-ed514a08b839 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:35:17.271321 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd76aa0-c191-4ff4-bef3-ed514a08b839 admin admin] REQUEST_METHOD: `PUT` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:35:17.271321 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd76aa0-c191-4ff4-bef3-ed514a08b839 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:35:17.271321 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd76aa0-c191-4ff4-bef3-ed514a08b839 admin admin] PATH_INFO: `/v3/projects/eee1cc17e9634449b3d2d1b8179aab59/users/c211020f686f4c8ca5874f00c853b963/roles/ea4c814713b149108e488de855b025e4` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:35:17.287674 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd76aa0-c191-4ff4-bef3-ed514a08b839 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eee1cc17e9634449b3d2d1b8179aab59, user_id=c211020f686f4c8ca5874f00c853b963, role_id=ea4c814713b149108e488de855b025e4)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:35:17.290423 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd76aa0-c191-4ff4-bef3-ed514a08b839 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:35:17.324235 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b734b4-9bda-4d4a-9af3-bc60ea9a4a86 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:35:17.325729 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b734b4-9bda-4d4a-9af3-bc60ea9a4a86 service neutron] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:35:17.336067 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 818/1622] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Sep 6 04:35:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 04:35:17.336716 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b734b4-9bda-4d4a-9af3-bc60ea9a4a86 service neutron] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:35:17.345417 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b734b4-9bda-4d4a-9af3-bc60ea9a4a86 service neutron] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:35:17.349111 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65202e6-04a2-4db4-97da-4260485c221f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90492440f6a6471e8392361c098428aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b707733ce65c42edbc1fd95ad0a1c06c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:35:17.349755 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65202e6-04a2-4db4-97da-4260485c221f service neutron] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:35:17.350114 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65202e6-04a2-4db4-97da-4260485c221f service neutron] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:35:17.350395 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65202e6-04a2-4db4-97da-4260485c221f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:35:17.360198 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65202e6-04a2-4db4-97da-4260485c221f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:35:17.364306 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65202e6-04a2-4db4-97da-4260485c221f service neutron] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:35:17.371887 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 805/1623] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Sep 6 04:35:17 2022] PUT /identity/v3/projects/eee1cc17e9634449b3d2d1b8179aab59/users/c211020f686f4c8ca5874f00c853b963/roles/ea4c814713b149108e488de855b025e4 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 04:35:17.373707 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 819/1624] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Sep 6 04:35:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 04:35:17.376034 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65202e6-04a2-4db4-97da-4260485c221f service neutron] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:35:17.376182 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd76aa0-c191-4ff4-bef3-ed514a08b839 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:35:17.390749 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:35:17.391129 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65202e6-04a2-4db4-97da-4260485c221f service neutron] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:35:17.433748 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:35:17.451903 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd76aa0-c191-4ff4-bef3-ed514a08b839 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:35:17.458784 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13728da-8a19-4745-9ac3-1bb909297cdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:35:17.459473 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13728da-8a19-4745-9ac3-1bb909297cdc admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:35:17.460743 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13728da-8a19-4745-9ac3-1bb909297cdc admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:35:17.460743 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13728da-8a19-4745-9ac3-1bb909297cdc admin admin] PATH_INFO: `/v3/projects` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:35:17.460984 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052a86d8-9d16-436a-839d-130554902868 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:35:17.461634 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b13728da-8a19-4745-9ac3-1bb909297cdc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:35:17.461809 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052a86d8-9d16-436a-839d-130554902868 admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:35:17.462160 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052a86d8-9d16-436a-839d-130554902868 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:35:17.462511 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052a86d8-9d16-436a-839d-130554902868 admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:35:17.463228 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052a86d8-9d16-436a-839d-130554902868 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:35:17.463861 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b13728da-8a19-4745-9ac3-1bb909297cdc admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:35:17.465806 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052a86d8-9d16-436a-839d-130554902868 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:35:17.476307 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 820/1625] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Sep 6 04:35:17 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 04:35:17.476992 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052a86d8-9d16-436a-839d-130554902868 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:35:17.488414 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052a86d8-9d16-436a-839d-130554902868 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:35:17.498587 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0afc632b-e6a7-423b-86fc-6b071e476494 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:35:17.498587 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afc632b-e6a7-423b-86fc-6b071e476494 admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:35:17.498587 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afc632b-e6a7-423b-86fc-6b071e476494 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:35:17.498587 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afc632b-e6a7-423b-86fc-6b071e476494 admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:35:17.499009 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0afc632b-e6a7-423b-86fc-6b071e476494 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:35:17.499009 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0afc632b-e6a7-423b-86fc-6b071e476494 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:35:17.507975 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 821/1626] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Sep 6 04:35:17 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 04:35:17.508781 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0afc632b-e6a7-423b-86fc-6b071e476494 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:35:17.518641 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0afc632b-e6a7-423b-86fc-6b071e476494 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:35:17.523406 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8ade56-80e3-4be6-913b-023a6d7d2fd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:35:17.527356 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8ade56-80e3-4be6-913b-023a6d7d2fd0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:35:17.527356 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8ade56-80e3-4be6-913b-023a6d7d2fd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:35:17.527356 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8ade56-80e3-4be6-913b-023a6d7d2fd0 admin admin] PATH_INFO: `/v3/projects/b4f8538b00954b3dacfd9ff9bead0e41/users/9df0ff1eb10946e192622a26876d1674/roles/ea4c814713b149108e488de855b025e4` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:35:17.527798 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a8ade56-80e3-4be6-913b-023a6d7d2fd0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b4f8538b00954b3dacfd9ff9bead0e41, user_id=9df0ff1eb10946e192622a26876d1674, role_id=ea4c814713b149108e488de855b025e4)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:35:17.530196 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a8ade56-80e3-4be6-913b-023a6d7d2fd0 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:35:17.542669 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 822/1627] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Sep 6 04:35:17 2022] PUT /identity/v3/projects/b4f8538b00954b3dacfd9ff9bead0e41/users/9df0ff1eb10946e192622a26876d1674/roles/ea4c814713b149108e488de855b025e4 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 04:35:17.543472 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8ade56-80e3-4be6-913b-023a6d7d2fd0 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:35:17.608462 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8ade56-80e3-4be6-913b-023a6d7d2fd0 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:35:17.614688 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b7acff-1037-4d97-a3a1-4d5948328870 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:35:17.618304 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 806/1628] 10.222.0.30 () {60 vars in 1153 bytes} [Tue Sep 6 04:35:17 2022] POST /identity/v3/projects => generated 385 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 04:35:17.618489 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b7acff-1037-4d97-a3a1-4d5948328870 admin admin] REQUEST_METHOD: `PUT` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:35:17.618489 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b7acff-1037-4d97-a3a1-4d5948328870 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:35:17.618489 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b7acff-1037-4d97-a3a1-4d5948328870 admin admin] PATH_INFO: `/v3/projects/eee1cc17e9634449b3d2d1b8179aab59/users/c211020f686f4c8ca5874f00c853b963/roles/ea4c814713b149108e488de855b025e4` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:35:17.618761 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fea2ca1-7531-40a1-9f8a-8fc18a16a254 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:35:17.618761 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fea2ca1-7531-40a1-9f8a-8fc18a16a254 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:35:17.618761 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fea2ca1-7531-40a1-9f8a-8fc18a16a254 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:35:17.621229 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b7acff-1037-4d97-a3a1-4d5948328870 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eee1cc17e9634449b3d2d1b8179aab59, user_id=c211020f686f4c8ca5874f00c853b963, role_id=ea4c814713b149108e488de855b025e4)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:35:17.624027 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b7acff-1037-4d97-a3a1-4d5948328870 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:35:17.637227 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 823/1629] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Sep 6 04:35:17 2022] PUT /identity/v3/projects/eee1cc17e9634449b3d2d1b8179aab59/users/c211020f686f4c8ca5874f00c853b963/roles/ea4c814713b149108e488de855b025e4 => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 04:35:17.638325 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b7acff-1037-4d97-a3a1-4d5948328870 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:35:17.655021 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-1fea2ca1-7531-40a1-9f8a-8fc18a16a254 None None] MFA Rules not processed for user `9df0ff1eb10946e192622a26876d1674`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:35:17.656406 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-1fea2ca1-7531-40a1-9f8a-8fc18a16a254 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:35:17.719973 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b7acff-1037-4d97-a3a1-4d5948328870 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:35:17.725747 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ff54c4-e374-4ae5-a664-1eacc53c2e66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:35:17.726618 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ff54c4-e374-4ae5-a664-1eacc53c2e66 admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:35:17.726901 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ff54c4-e374-4ae5-a664-1eacc53c2e66 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:35:17.727179 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ff54c4-e374-4ae5-a664-1eacc53c2e66 admin admin] PATH_INFO: `/v3/users` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:35:17.727848 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ff54c4-e374-4ae5-a664-1eacc53c2e66 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:35:17.731599 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ff54c4-e374-4ae5-a664-1eacc53c2e66 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:35:17.751136 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 807/1630] 10.222.0.30 () {58 vars in 958 bytes} [Tue Sep 6 04:35:17 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 04:35:17.759952 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdaa92d-fac0-4a30-aa97-b42270864315 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:35:17.759952 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdaa92d-fac0-4a30-aa97-b42270864315 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:35:17.759952 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdaa92d-fac0-4a30-aa97-b42270864315 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:35:17.794512 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-5cdaa92d-fac0-4a30-aa97-b42270864315 None None] MFA Rules not processed for user `c211020f686f4c8ca5874f00c853b963`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:35:17.796010 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-5cdaa92d-fac0-4a30-aa97-b42270864315 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:35:17.874382 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 824/1631] 10.222.0.30 () {60 vars in 1144 bytes} [Tue Sep 6 04:35:17 2022] POST /identity/v3/users => generated 293 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 04:35:17.875585 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ff54c4-e374-4ae5-a664-1eacc53c2e66 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:35:17.881559 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 808/1632] 10.222.0.30 () {58 vars in 958 bytes} [Tue Sep 6 04:35:17 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 04:35:17.881559 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cdaa92d-fac0-4a30-aa97-b42270864315 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:35:17.902565 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cdaa92d-fac0-4a30-aa97-b42270864315 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:35:17.908948 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1e9662-f655-49ee-9c3f-c281264795d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:35:17.909891 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1e9662-f655-49ee-9c3f-c281264795d1 admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:35:17.911559 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1e9662-f655-49ee-9c3f-c281264795d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:35:17.911559 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1e9662-f655-49ee-9c3f-c281264795d1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:35:17.912309 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1e9662-f655-49ee-9c3f-c281264795d1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:35:17.913970 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1e9662-f655-49ee-9c3f-c281264795d1 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:35:17.915241 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:35:17.918965 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:35:17.929559 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 809/1633] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Sep 6 04:35:17 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 04:35:17.930665 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1e9662-f655-49ee-9c3f-c281264795d1 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:35:17.947474 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ff54c4-e374-4ae5-a664-1eacc53c2e66 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:35:17.952811 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7cfeb9-c221-48a5-bd3b-07a876b56739 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd09b5ab675634a2eb4b03755743f7d0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b707733ce65c42edbc1fd95ad0a1c06c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:35:17.953526 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7cfeb9-c221-48a5-bd3b-07a876b56739 service cinder] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:35:17.953800 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7cfeb9-c221-48a5-bd3b-07a876b56739 service cinder] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:35:17.954077 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7cfeb9-c221-48a5-bd3b-07a876b56739 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:35:17.967071 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7cfeb9-c221-48a5-bd3b-07a876b56739 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:35:17.968955 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7cfeb9-c221-48a5-bd3b-07a876b56739 service cinder] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:35:17.987646 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 825/1634] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Sep 6 04:35:17 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 04:35:17.989284 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7cfeb9-c221-48a5-bd3b-07a876b56739 service cinder] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:35:18.017426 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7cfeb9-c221-48a5-bd3b-07a876b56739 service cinder] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:35:18.022482 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1e9662-f655-49ee-9c3f-c281264795d1 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:35:18.036690 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57551fa2-4520-44ff-865c-b907eda7bfca service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd09b5ab675634a2eb4b03755743f7d0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b707733ce65c42edbc1fd95ad0a1c06c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:35:18.037494 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57551fa2-4520-44ff-865c-b907eda7bfca service cinder] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:35:18.037854 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57551fa2-4520-44ff-865c-b907eda7bfca service cinder] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:35:18.038209 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57551fa2-4520-44ff-865c-b907eda7bfca service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:35:18.042840 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1478d7a-91b4-4910-8bc9-3ca20035a1c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:35:18.046682 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1478d7a-91b4-4910-8bc9-3ca20035a1c0 admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:35:18.046968 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1478d7a-91b4-4910-8bc9-3ca20035a1c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:35:18.050169 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1478d7a-91b4-4910-8bc9-3ca20035a1c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:35:18.051157 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1478d7a-91b4-4910-8bc9-3ca20035a1c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:35:18.056608 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1478d7a-91b4-4910-8bc9-3ca20035a1c0 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:35:18.059024 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57551fa2-4520-44ff-865c-b907eda7bfca service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:35:18.064238 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57551fa2-4520-44ff-865c-b907eda7bfca service cinder] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:35:18.076941 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 826/1635] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Sep 6 04:35:17 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 04:35:18.092853 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1478d7a-91b4-4910-8bc9-3ca20035a1c0 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:35:18.094631 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 810/1636] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Sep 6 04:35:17 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 04:35:18.108047 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1478d7a-91b4-4910-8bc9-3ca20035a1c0 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:35:18.112958 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17a8d0e-c4b9-4d82-95c6-d7f9cbf53840 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:35:18.113651 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17a8d0e-c4b9-4d82-95c6-d7f9cbf53840 admin admin] REQUEST_METHOD: `PUT` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:35:18.113934 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17a8d0e-c4b9-4d82-95c6-d7f9cbf53840 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:35:18.114212 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17a8d0e-c4b9-4d82-95c6-d7f9cbf53840 admin admin] PATH_INFO: `/v3/projects/95bedd195b214bfab139a44b3e770e0c/users/44f40ae47fd2460282a63de44e415cce/roles/29607b2de39f40d68bb6ceef64b8a732` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:35:18.131905 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17a8d0e-c4b9-4d82-95c6-d7f9cbf53840 admin admin] RBAC: Authorizing `identity:create_grant(project_id=95bedd195b214bfab139a44b3e770e0c, user_id=44f40ae47fd2460282a63de44e415cce, role_id=29607b2de39f40d68bb6ceef64b8a732)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:35:18.134862 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17a8d0e-c4b9-4d82-95c6-d7f9cbf53840 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:35:18.256051 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 827/1637] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Sep 6 04:35:18 2022] PUT /identity/v3/projects/95bedd195b214bfab139a44b3e770e0c/users/44f40ae47fd2460282a63de44e415cce/roles/29607b2de39f40d68bb6ceef64b8a732 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 04:35:18.256764 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76608c8-6922-4cc5-907e-0cecfe18cb1b None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:35:18.257299 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76608c8-6922-4cc5-907e-0cecfe18cb1b None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:35:18.257481 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76608c8-6922-4cc5-907e-0cecfe18cb1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:35:18.279055 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9336414b-073c-4e09-8da3-2870b60f1523 None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:35:18.279651 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9336414b-073c-4e09-8da3-2870b60f1523 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:35:18.280187 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9336414b-073c-4e09-8da3-2870b60f1523 None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:35:18.281899 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 828/1638] 10.222.0.30 () {54 vars in 962 bytes} [Tue Sep 6 04:35:18 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 06 04:35:18.289350 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f85ca1f-ceb6-482b-88d3-df0e89229a3f None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:35:18.289952 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f85ca1f-ceb6-482b-88d3-df0e89229a3f None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:35:18.290719 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f85ca1f-ceb6-482b-88d3-df0e89229a3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:35:18.319066 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-b76608c8-6922-4cc5-907e-0cecfe18cb1b None None] MFA Rules not processed for user `44f40ae47fd2460282a63de44e415cce`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:35:18.320339 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-b76608c8-6922-4cc5-907e-0cecfe18cb1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:35:18.358335 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-2f85ca1f-ceb6-482b-88d3-df0e89229a3f None None] MFA Rules not processed for user `9db138b77afa4b068244b8ef715404fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:35:18.360586 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-2f85ca1f-ceb6-482b-88d3-df0e89229a3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:35:18.509459 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 811/1639] 10.222.0.30 () {58 vars in 958 bytes} [Tue Sep 6 04:35:18 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 254 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 04:35:18.512294 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76608c8-6922-4cc5-907e-0cecfe18cb1b None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:35:18.520049 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 829/1640] 10.222.0.30 () {58 vars in 1059 bytes} [Tue Sep 6 04:35:18 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 04:35:18.521057 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f85ca1f-ceb6-482b-88d3-df0e89229a3f None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:35:18.601737 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f85ca1f-ceb6-482b-88d3-df0e89229a3f None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:35:18.608757 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26858d86-5b8f-435a-8400-fd92f1ba1436 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd09b5ab675634a2eb4b03755743f7d0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b707733ce65c42edbc1fd95ad0a1c06c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:35:18.609464 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26858d86-5b8f-435a-8400-fd92f1ba1436 service cinder] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:35:18.609823 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26858d86-5b8f-435a-8400-fd92f1ba1436 service cinder] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:35:18.610103 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26858d86-5b8f-435a-8400-fd92f1ba1436 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:35:18.675870 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76608c8-6922-4cc5-907e-0cecfe18cb1b None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:35:18.682704 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2138dde-288c-440c-91bd-72ed2384fe15 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd09b5ab675634a2eb4b03755743f7d0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b707733ce65c42edbc1fd95ad0a1c06c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:35:18.683750 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2138dde-288c-440c-91bd-72ed2384fe15 service cinder] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:35:18.684041 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2138dde-288c-440c-91bd-72ed2384fe15 service cinder] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:35:18.686217 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2138dde-288c-440c-91bd-72ed2384fe15 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:35:18.687740 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:35:18.768239 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:35:18.770642 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26858d86-5b8f-435a-8400-fd92f1ba1436 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:35:18.773347 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26858d86-5b8f-435a-8400-fd92f1ba1436 service cinder] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:35:18.786054 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 830/1641] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Sep 6 04:35:18 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 265 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 04:35:18.786949 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26858d86-5b8f-435a-8400-fd92f1ba1436 service cinder] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:35:18.787763 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2138dde-288c-440c-91bd-72ed2384fe15 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:35:18.789744 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2138dde-288c-440c-91bd-72ed2384fe15 service cinder] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:35:18.803360 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 812/1642] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Sep 6 04:35:18 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 04:35:18.805059 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f00ed7-8b14-4280-bce9-6bafd941b48f None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:35:18.805710 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f00ed7-8b14-4280-bce9-6bafd941b48f None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:35:18.812636 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f00ed7-8b14-4280-bce9-6bafd941b48f None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:35:18.812636 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 813/1643] 10.222.0.30 () {54 vars in 912 bytes} [Tue Sep 6 04:35:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 04:35:18.818143 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01d166a-0129-44ac-9595-6ba77a3547f4 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:35:18.822616 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01d166a-0129-44ac-9595-6ba77a3547f4 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:35:18.826433 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01d166a-0129-44ac-9595-6ba77a3547f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:35:18.857414 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-b01d166a-0129-44ac-9595-6ba77a3547f4 None None] MFA Rules not processed for user `9db138b77afa4b068244b8ef715404fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:35:18.858724 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-b01d166a-0129-44ac-9595-6ba77a3547f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:35:18.872503 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 814/1644] 10.222.0.30 () {58 vars in 1009 bytes} [Tue Sep 6 04:35:18 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 04:35:18.993874 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26858d86-5b8f-435a-8400-fd92f1ba1436 service cinder] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:35:18.999555 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9248d5-d55f-4e95-bf00-2dfe15034db8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:35:19.000412 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9248d5-d55f-4e95-bf00-2dfe15034db8 admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:35:19.000563 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9248d5-d55f-4e95-bf00-2dfe15034db8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:35:19.000731 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9248d5-d55f-4e95-bf00-2dfe15034db8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:35:19.001411 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9248d5-d55f-4e95-bf00-2dfe15034db8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:35:19.003772 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9248d5-d55f-4e95-bf00-2dfe15034db8 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:35:19.180353 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 831/1645] 10.222.0.30 () {60 vars in 1153 bytes} [Tue Sep 6 04:35:18 2022] POST /identity/v3/projects => generated 385 bytes in 394 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 04:35:19.184537 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01d166a-0129-44ac-9595-6ba77a3547f4 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:35:19.196735 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01d166a-0129-44ac-9595-6ba77a3547f4 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:35:19.268172 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15724c0-a217-4893-b1cd-78cef522027a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:35:19.268778 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15724c0-a217-4893-b1cd-78cef522027a admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:35:19.268778 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15724c0-a217-4893-b1cd-78cef522027a admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:35:19.269214 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15724c0-a217-4893-b1cd-78cef522027a admin admin] PATH_INFO: `/v3/users` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:35:19.270320 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15724c0-a217-4893-b1cd-78cef522027a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:35:19.272512 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15724c0-a217-4893-b1cd-78cef522027a admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:35:19.415053 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 815/1646] 10.222.0.30 () {60 vars in 1144 bytes} [Tue Sep 6 04:35:19 2022] POST /identity/v3/users => generated 299 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 04:35:19.671144 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9248d5-d55f-4e95-bf00-2dfe15034db8 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:35:19.675478 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15724c0-a217-4893-b1cd-78cef522027a admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:35:19.675995 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9248d5-d55f-4e95-bf00-2dfe15034db8 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:35:19.675995 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68147593-7ada-4d9e-a245-8230ad6d1843 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:35:19.675995 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68147593-7ada-4d9e-a245-8230ad6d1843 admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:35:19.675995 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68147593-7ada-4d9e-a245-8230ad6d1843 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:35:19.676603 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68147593-7ada-4d9e-a245-8230ad6d1843 admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:35:19.676603 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68147593-7ada-4d9e-a245-8230ad6d1843 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:35:19.676603 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68147593-7ada-4d9e-a245-8230ad6d1843 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:35:19.676603 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 832/1647] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Sep 6 04:35:19 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 04:35:19.676603 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68147593-7ada-4d9e-a245-8230ad6d1843 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:35:19.676603 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68147593-7ada-4d9e-a245-8230ad6d1843 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:35:19.677755 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82fb254e-4ffa-4b7e-b659-0dbbe5c885ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:35:19.677755 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fb254e-4ffa-4b7e-b659-0dbbe5c885ea admin admin] REQUEST_METHOD: `PUT` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:35:19.677755 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fb254e-4ffa-4b7e-b659-0dbbe5c885ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:35:19.677755 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fb254e-4ffa-4b7e-b659-0dbbe5c885ea admin admin] PATH_INFO: `/v3/projects/755030d1ad9d4945b341d08a3af30d1a/users/bdef1c0f68b34452877b2123825d6dd2/roles/ea4c814713b149108e488de855b025e4` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:35:19.700166 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15724c0-a217-4893-b1cd-78cef522027a admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:35:19.700166 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6dfcf27-5666-40ff-b15d-7447e7bfe0c5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd09b5ab675634a2eb4b03755743f7d0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b707733ce65c42edbc1fd95ad0a1c06c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:35:19.700166 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6dfcf27-5666-40ff-b15d-7447e7bfe0c5 service cinder] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:35:19.700166 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6dfcf27-5666-40ff-b15d-7447e7bfe0c5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:35:19.701402 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6dfcf27-5666-40ff-b15d-7447e7bfe0c5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:35:19.701787 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82fb254e-4ffa-4b7e-b659-0dbbe5c885ea admin admin] RBAC: Authorizing `identity:create_grant(project_id=755030d1ad9d4945b341d08a3af30d1a, user_id=bdef1c0f68b34452877b2123825d6dd2, role_id=ea4c814713b149108e488de855b025e4)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:35:19.701787 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82fb254e-4ffa-4b7e-b659-0dbbe5c885ea admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:35:19.721750 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6dfcf27-5666-40ff-b15d-7447e7bfe0c5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:35:19.723210 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6dfcf27-5666-40ff-b15d-7447e7bfe0c5 service cinder] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:35:19.736216 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 816/1648] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Sep 6 04:35:19 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 04:35:19.827925 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 833/1649] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Sep 6 04:35:19 2022] PUT /identity/v3/projects/755030d1ad9d4945b341d08a3af30d1a/users/bdef1c0f68b34452877b2123825d6dd2/roles/ea4c814713b149108e488de855b025e4 => generated 0 bytes in 327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 04:35:19.839865 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6dfcf27-5666-40ff-b15d-7447e7bfe0c5 service cinder] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:35:19.864453 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6dfcf27-5666-40ff-b15d-7447e7bfe0c5 service cinder] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:35:19.875623 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:35:20.016845 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507b9666-57a3-4657-93df-65d6b3ecdfa2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:35:20.018516 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507b9666-57a3-4657-93df-65d6b3ecdfa2 admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:35:20.018768 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507b9666-57a3-4657-93df-65d6b3ecdfa2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:35:20.021770 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507b9666-57a3-4657-93df-65d6b3ecdfa2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:35:20.021770 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507b9666-57a3-4657-93df-65d6b3ecdfa2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:35:20.022895 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507b9666-57a3-4657-93df-65d6b3ecdfa2 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:35:20.046658 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 817/1650] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Sep 6 04:35:19 2022] GET /identity/v3/roles => generated 1420 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 04:35:20.054107 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82fb254e-4ffa-4b7e-b659-0dbbe5c885ea admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:35:20.066225 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82fb254e-4ffa-4b7e-b659-0dbbe5c885ea admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:35:20.077251 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67dab17f-2745-4480-ac0e-42df61d261d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:35:20.078033 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67dab17f-2745-4480-ac0e-42df61d261d3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:35:20.078313 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67dab17f-2745-4480-ac0e-42df61d261d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:35:20.078583 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67dab17f-2745-4480-ac0e-42df61d261d3 admin admin] PATH_INFO: `/v3/projects/755030d1ad9d4945b341d08a3af30d1a/users/bdef1c0f68b34452877b2123825d6dd2/roles/ea4c814713b149108e488de855b025e4` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:35:20.083323 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67dab17f-2745-4480-ac0e-42df61d261d3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=755030d1ad9d4945b341d08a3af30d1a, user_id=bdef1c0f68b34452877b2123825d6dd2, role_id=ea4c814713b149108e488de855b025e4)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:35:20.086083 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67dab17f-2745-4480-ac0e-42df61d261d3 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:35:20.099763 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 834/1651] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Sep 6 04:35:20 2022] PUT /identity/v3/projects/755030d1ad9d4945b341d08a3af30d1a/users/bdef1c0f68b34452877b2123825d6dd2/roles/ea4c814713b149108e488de855b025e4 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 04:35:20.111989 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2b8aa4-1dae-444b-a448-3d734342c2a6 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:35:20.112277 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2b8aa4-1dae-444b-a448-3d734342c2a6 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:35:20.112552 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2b8aa4-1dae-444b-a448-3d734342c2a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:35:20.200249 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-af2b8aa4-1dae-444b-a448-3d734342c2a6 None None] MFA Rules not processed for user `bdef1c0f68b34452877b2123825d6dd2`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:35:20.204893 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-af2b8aa4-1dae-444b-a448-3d734342c2a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:35:20.389576 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 818/1652] 10.222.0.30 () {58 vars in 958 bytes} [Tue Sep 6 04:35:20 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 278 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 04:35:20.404590 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67dab17f-2745-4480-ac0e-42df61d261d3 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:35:20.478619 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67dab17f-2745-4480-ac0e-42df61d261d3 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:35:20.483771 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468e9848-f100-4d8c-a60d-b792b244877c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd09b5ab675634a2eb4b03755743f7d0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b707733ce65c42edbc1fd95ad0a1c06c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:35:20.488112 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468e9848-f100-4d8c-a60d-b792b244877c service cinder] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:35:20.488112 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468e9848-f100-4d8c-a60d-b792b244877c service cinder] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:35:20.488112 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468e9848-f100-4d8c-a60d-b792b244877c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:35:20.497823 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468e9848-f100-4d8c-a60d-b792b244877c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:35:20.499686 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468e9848-f100-4d8c-a60d-b792b244877c service cinder] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:35:20.522824 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 835/1653] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Sep 6 04:35:20 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 04:35:22.644225 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd72371e-f26e-4a8e-81f0-6ca3b3b836e0 None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:35:22.645066 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd72371e-f26e-4a8e-81f0-6ca3b3b836e0 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:35:22.645532 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd72371e-f26e-4a8e-81f0-6ca3b3b836e0 None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:35:22.647173 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 819/1654] 10.222.0.30 () {54 vars in 962 bytes} [Tue Sep 6 04:35:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 04:35:22.655349 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe0998b-8e03-4098-b69c-cdbf381b1b26 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:35:22.656545 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe0998b-8e03-4098-b69c-cdbf381b1b26 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:35:22.656966 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe0998b-8e03-4098-b69c-cdbf381b1b26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:35:22.711152 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-cfe0998b-8e03-4098-b69c-cdbf381b1b26 None None] MFA Rules not processed for user `9db138b77afa4b068244b8ef715404fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:35:22.712850 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-cfe0998b-8e03-4098-b69c-cdbf381b1b26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:35:22.735456 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:35:22.894814 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 836/1655] 10.222.0.30 () {58 vars in 1059 bytes} [Tue Sep 6 04:35:22 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 04:35:22.901343 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d914da84-1d9e-4083-94d7-6da2e5609bb9 None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:35:22.903541 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d914da84-1d9e-4083-94d7-6da2e5609bb9 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:35:22.903541 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d914da84-1d9e-4083-94d7-6da2e5609bb9 None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:35:22.903873 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 820/1656] 10.222.0.30 () {54 vars in 912 bytes} [Tue Sep 6 04:35:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 04:35:22.915920 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e030c18-b527-456a-adfd-0c42c07fa286 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:35:22.916246 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e030c18-b527-456a-adfd-0c42c07fa286 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:35:22.916517 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e030c18-b527-456a-adfd-0c42c07fa286 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:35:22.958519 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-0e030c18-b527-456a-adfd-0c42c07fa286 None None] MFA Rules not processed for user `9db138b77afa4b068244b8ef715404fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:35:22.960948 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-0e030c18-b527-456a-adfd-0c42c07fa286 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:35:22.974610 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 837/1657] 10.222.0.30 () {58 vars in 1009 bytes} [Tue Sep 6 04:35:22 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 04:35:23.000938 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d914da84-1d9e-4083-94d7-6da2e5609bb9 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:35:23.017439 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d914da84-1d9e-4083-94d7-6da2e5609bb9 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:35:23.024664 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce28b195-748f-4858-a312-a650669c9ee0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd09b5ab675634a2eb4b03755743f7d0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b707733ce65c42edbc1fd95ad0a1c06c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:35:23.026393 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce28b195-748f-4858-a312-a650669c9ee0 service cinder] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:35:23.029994 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce28b195-748f-4858-a312-a650669c9ee0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:35:23.029994 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce28b195-748f-4858-a312-a650669c9ee0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:35:23.048549 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce28b195-748f-4858-a312-a650669c9ee0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:35:23.050575 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce28b195-748f-4858-a312-a650669c9ee0 service cinder] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:35:23.079415 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 821/1658] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Sep 6 04:35:22 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 74 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 04:35:25.411810 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e030c18-b527-456a-adfd-0c42c07fa286 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:35:25.422068 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e030c18-b527-456a-adfd-0c42c07fa286 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:35:25.427108 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053899a9-4689-4c49-9f7d-d9274dce80d0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd09b5ab675634a2eb4b03755743f7d0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b707733ce65c42edbc1fd95ad0a1c06c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:35:25.430029 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053899a9-4689-4c49-9f7d-d9274dce80d0 service cinder] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:35:25.430029 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053899a9-4689-4c49-9f7d-d9274dce80d0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:35:25.430029 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053899a9-4689-4c49-9f7d-d9274dce80d0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:35:25.501071 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-053899a9-4689-4c49-9f7d-d9274dce80d0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:35:25.503191 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-053899a9-4689-4c49-9f7d-d9274dce80d0 service cinder] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:35:25.526686 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 838/1659] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Sep 6 04:35:25 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 04:35:29.768774 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757e5f10-4a61-45b6-86de-183b78422b6b None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:35:29.769227 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757e5f10-4a61-45b6-86de-183b78422b6b None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:35:29.769227 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757e5f10-4a61-45b6-86de-183b78422b6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:35:29.816185 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-757e5f10-4a61-45b6-86de-183b78422b6b None None] MFA Rules not processed for user `5a0294ea40914bf290902307a787642d`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:35:29.818952 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-757e5f10-4a61-45b6-86de-183b78422b6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:35:30.017145 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 822/1660] 10.222.0.30 () {58 vars in 958 bytes} [Tue Sep 6 04:35:29 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 250 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 04:35:30.043539 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053899a9-4689-4c49-9f7d-d9274dce80d0 service cinder] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:35:30.075354 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053899a9-4689-4c49-9f7d-d9274dce80d0 service cinder] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:35:30.079308 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2bbaef4-3d11-449b-8d7b-be30689b4b5d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9db138b77afa4b068244b8ef715404fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b707733ce65c42edbc1fd95ad0a1c06c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:35:30.080443 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2bbaef4-3d11-449b-8d7b-be30689b4b5d service nova] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:35:30.080742 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2bbaef4-3d11-449b-8d7b-be30689b4b5d service nova] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:35:30.081024 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2bbaef4-3d11-449b-8d7b-be30689b4b5d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:35:30.105444 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2bbaef4-3d11-449b-8d7b-be30689b4b5d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:35:30.108866 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2bbaef4-3d11-449b-8d7b-be30689b4b5d service nova] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:35:30.136549 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 839/1661] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Sep 6 04:35:30 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 04:35:30.502381 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c2efe7-1720-4345-ba4d-4ec0301b71aa None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:35:30.506861 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c2efe7-1720-4345-ba4d-4ec0301b71aa None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:35:30.506861 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c2efe7-1720-4345-ba4d-4ec0301b71aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:35:30.531840 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:35:30.580803 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-a3c2efe7-1720-4345-ba4d-4ec0301b71aa None None] MFA Rules not processed for user `dec1c17550b849dbadbc16e98530b455`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:35:30.582200 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-a3c2efe7-1720-4345-ba4d-4ec0301b71aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:35:30.734974 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeec9593-94b4-4e5f-bb3a-2f129eea0aaf None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:35:30.735554 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeec9593-94b4-4e5f-bb3a-2f129eea0aaf None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:35:30.736509 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeec9593-94b4-4e5f-bb3a-2f129eea0aaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:35:30.738291 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 823/1662] 10.222.0.30 () {58 vars in 958 bytes} [Tue Sep 6 04:35:30 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 04:35:30.797737 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-eeec9593-94b4-4e5f-bb3a-2f129eea0aaf None None] MFA Rules not processed for user `dec1c17550b849dbadbc16e98530b455`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:35:30.799132 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-eeec9593-94b4-4e5f-bb3a-2f129eea0aaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:35:30.824497 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 840/1663] 10.222.0.30 () {58 vars in 958 bytes} [Tue Sep 6 04:35:30 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 04:35:30.837583 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c2efe7-1720-4345-ba4d-4ec0301b71aa None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:35:30.860032 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c2efe7-1720-4345-ba4d-4ec0301b71aa None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:35:30.866590 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1600dfbf-0f78-457a-8e53-4089057e0896 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9db138b77afa4b068244b8ef715404fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b707733ce65c42edbc1fd95ad0a1c06c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:35:30.869046 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1600dfbf-0f78-457a-8e53-4089057e0896 service nova] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:35:30.869046 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1600dfbf-0f78-457a-8e53-4089057e0896 service nova] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:35:30.869046 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1600dfbf-0f78-457a-8e53-4089057e0896 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:35:30.884592 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1600dfbf-0f78-457a-8e53-4089057e0896 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:35:30.888825 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1600dfbf-0f78-457a-8e53-4089057e0896 service nova] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:35:30.907942 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 824/1664] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Sep 6 04:35:30 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 71 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 04:35:31.741428 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab56a996-b931-47e8-8811-9834b51df3ba None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:35:31.742292 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab56a996-b931-47e8-8811-9834b51df3ba None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:35:31.742737 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab56a996-b931-47e8-8811-9834b51df3ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:35:31.819337 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-ab56a996-b931-47e8-8811-9834b51df3ba None None] MFA Rules not processed for user `1ca147cfc97149dd87e533123a262cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:35:31.820730 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-ab56a996-b931-47e8-8811-9834b51df3ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:35:31.991205 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 841/1665] 10.222.0.30 () {58 vars in 958 bytes} [Tue Sep 6 04:35:31 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 251 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 04:35:31.992995 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1600dfbf-0f78-457a-8e53-4089057e0896 service nova] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:35:32.008577 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1600dfbf-0f78-457a-8e53-4089057e0896 service nova] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:35:32.025145 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0918ed7d-87d9-437f-9dc0-a5d969172a4c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9db138b77afa4b068244b8ef715404fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b707733ce65c42edbc1fd95ad0a1c06c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:35:32.026007 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0918ed7d-87d9-437f-9dc0-a5d969172a4c service nova] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:35:32.028663 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0918ed7d-87d9-437f-9dc0-a5d969172a4c service nova] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:35:32.031515 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0918ed7d-87d9-437f-9dc0-a5d969172a4c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:35:32.052222 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0918ed7d-87d9-437f-9dc0-a5d969172a4c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:35:32.055038 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0918ed7d-87d9-437f-9dc0-a5d969172a4c service nova] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:35:32.071499 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130d9cdd-ba98-4d53-b03d-d67df654a401 None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:35:32.071800 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130d9cdd-ba98-4d53-b03d-d67df654a401 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:35:32.072085 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130d9cdd-ba98-4d53-b03d-d67df654a401 None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:35:32.074367 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 842/1666] 10.222.0.30 () {54 vars in 912 bytes} [Tue Sep 6 04:35:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 04:35:32.079939 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca9e831-ff9f-4dbb-84f1-19b78804b85d None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:35:32.081477 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca9e831-ff9f-4dbb-84f1-19b78804b85d None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:35:32.081945 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca9e831-ff9f-4dbb-84f1-19b78804b85d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:35:32.083915 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 825/1667] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Sep 6 04:35:31 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 04:35:32.116240 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:35:32.225242 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-0ca9e831-ff9f-4dbb-84f1-19b78804b85d None None] MFA Rules not processed for user `90492440f6a6471e8392361c098428aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:35:32.227127 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-0ca9e831-ff9f-4dbb-84f1-19b78804b85d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:35:32.376217 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 843/1668] 10.222.0.30 () {58 vars in 1009 bytes} [Tue Sep 6 04:35:32 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 297 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 04:35:32.409190 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0918ed7d-87d9-437f-9dc0-a5d969172a4c service nova] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:35:32.427554 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0918ed7d-87d9-437f-9dc0-a5d969172a4c service nova] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:35:32.434958 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271a4ca6-812c-4ed3-9734-bc31f6c5f6a8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90492440f6a6471e8392361c098428aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b707733ce65c42edbc1fd95ad0a1c06c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:35:32.438735 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271a4ca6-812c-4ed3-9734-bc31f6c5f6a8 service neutron] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:35:32.439203 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271a4ca6-812c-4ed3-9734-bc31f6c5f6a8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:35:32.439639 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271a4ca6-812c-4ed3-9734-bc31f6c5f6a8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:35:32.462839 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-271a4ca6-812c-4ed3-9734-bc31f6c5f6a8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:35:32.464581 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-271a4ca6-812c-4ed3-9734-bc31f6c5f6a8 service neutron] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:35:32.482152 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 826/1669] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Sep 6 04:35:32 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 73 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 04:35:33.206015 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af34f59b-1f29-4755-a9ae-e4fa5bc7b719 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:35:33.206533 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af34f59b-1f29-4755-a9ae-e4fa5bc7b719 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:35:33.206533 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af34f59b-1f29-4755-a9ae-e4fa5bc7b719 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:35:33.285908 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-af34f59b-1f29-4755-a9ae-e4fa5bc7b719 None None] MFA Rules not processed for user `15eebbd8b1d44f4897acb266633998db`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:35:33.287894 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-af34f59b-1f29-4755-a9ae-e4fa5bc7b719 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:35:33.440086 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 844/1670] 10.222.0.30 () {58 vars in 958 bytes} [Tue Sep 6 04:35:33 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 04:35:33.442067 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6c17a2-21be-4b86-97d6-69d6ec97a5eb None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:35:33.447690 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6c17a2-21be-4b86-97d6-69d6ec97a5eb None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:35:33.447690 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6c17a2-21be-4b86-97d6-69d6ec97a5eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:35:33.514633 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-8f6c17a2-21be-4b86-97d6-69d6ec97a5eb None None] MFA Rules not processed for user `15eebbd8b1d44f4897acb266633998db`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:35:33.516275 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-8f6c17a2-21be-4b86-97d6-69d6ec97a5eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:35:33.528328 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 827/1671] 10.222.0.30 () {58 vars in 958 bytes} [Tue Sep 6 04:35:33 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 04:35:33.549647 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af34f59b-1f29-4755-a9ae-e4fa5bc7b719 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:35:33.576537 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af34f59b-1f29-4755-a9ae-e4fa5bc7b719 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:35:33.586647 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac6de73-dae9-42b8-ae31-a0a219b23ea3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9db138b77afa4b068244b8ef715404fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b707733ce65c42edbc1fd95ad0a1c06c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:35:33.587389 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac6de73-dae9-42b8-ae31-a0a219b23ea3 service nova] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:35:33.587744 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac6de73-dae9-42b8-ae31-a0a219b23ea3 service nova] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:35:33.588098 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac6de73-dae9-42b8-ae31-a0a219b23ea3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:35:33.629366 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac6de73-dae9-42b8-ae31-a0a219b23ea3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:35:33.629366 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac6de73-dae9-42b8-ae31-a0a219b23ea3 service nova] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:35:33.668048 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 845/1672] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Sep 6 04:35:33 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 04:35:33.798190 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6c17a2-21be-4b86-97d6-69d6ec97a5eb None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:35:33.897536 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6c17a2-21be-4b86-97d6-69d6ec97a5eb None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:35:33.906509 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f89264b-350f-4d34-a8cb-cbc7862ba4e3 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d4f2e4dbd41441d84f03ddb30191e7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b707733ce65c42edbc1fd95ad0a1c06c', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:35:33.907311 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f89264b-350f-4d34-a8cb-cbc7862ba4e3 service placement] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:35:33.908569 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f89264b-350f-4d34-a8cb-cbc7862ba4e3 service placement] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:35:33.909024 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f89264b-350f-4d34-a8cb-cbc7862ba4e3 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:35:33.922964 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-3f89264b-350f-4d34-a8cb-cbc7862ba4e3 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:35:33.984752 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f89264b-350f-4d34-a8cb-cbc7862ba4e3 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:35:33.986419 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f89264b-350f-4d34-a8cb-cbc7862ba4e3 service placement] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:35:33.994497 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc941ad-881e-4622-9c45-5c1c0d249d79 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:35:33.994793 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc941ad-881e-4622-9c45-5c1c0d249d79 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:35:33.995553 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc941ad-881e-4622-9c45-5c1c0d249d79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:35:34.010867 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 828/1673] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Sep 6 04:35:33 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 213 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 04:35:34.032430 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-3cc941ad-881e-4622-9c45-5c1c0d249d79 None None] MFA Rules not processed for user `dec1c17550b849dbadbc16e98530b455`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:35:34.038209 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-3cc941ad-881e-4622-9c45-5c1c0d249d79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:35:34.062895 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 846/1674] 10.222.0.30 () {58 vars in 958 bytes} [Tue Sep 6 04:35:33 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 70 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 04:35:34.695163 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98063500-6cb7-4381-8109-249f6cbb32dd None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:35:34.695648 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98063500-6cb7-4381-8109-249f6cbb32dd None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:35:34.695738 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98063500-6cb7-4381-8109-249f6cbb32dd None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:35:34.697050 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 829/1675] 10.222.0.30 () {54 vars in 912 bytes} [Tue Sep 6 04:35:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 04:35:34.703968 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193eaea0-1a2e-4b8c-b52b-9e5f8a840ffe None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:35:34.704546 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193eaea0-1a2e-4b8c-b52b-9e5f8a840ffe None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:35:34.704546 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193eaea0-1a2e-4b8c-b52b-9e5f8a840ffe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:35:34.743471 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-193eaea0-1a2e-4b8c-b52b-9e5f8a840ffe None None] MFA Rules not processed for user `90492440f6a6471e8392361c098428aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:35:34.746948 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-193eaea0-1a2e-4b8c-b52b-9e5f8a840ffe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:35:34.761692 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 847/1676] 10.222.0.30 () {58 vars in 1009 bytes} [Tue Sep 6 04:35:34 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 04:35:34.783959 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98063500-6cb7-4381-8109-249f6cbb32dd None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:35:34.815248 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98063500-6cb7-4381-8109-249f6cbb32dd None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:35:34.828482 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1cccf9b-c6bc-41ba-a3a7-7afb64781ea6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90492440f6a6471e8392361c098428aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b707733ce65c42edbc1fd95ad0a1c06c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:35:34.833943 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cccf9b-c6bc-41ba-a3a7-7afb64781ea6 service neutron] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:35:34.834575 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cccf9b-c6bc-41ba-a3a7-7afb64781ea6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:35:34.835022 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cccf9b-c6bc-41ba-a3a7-7afb64781ea6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:35:34.852473 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1cccf9b-c6bc-41ba-a3a7-7afb64781ea6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:35:34.856526 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1cccf9b-c6bc-41ba-a3a7-7afb64781ea6 service neutron] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:35:34.865178 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 830/1677] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Sep 6 04:35:34 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 04:35:36.333775 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250885a9-f7fd-40c1-a3fd-fe6241337751 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:35:36.334410 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250885a9-f7fd-40c1-a3fd-fe6241337751 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:35:36.334828 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250885a9-f7fd-40c1-a3fd-fe6241337751 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:35:36.362080 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-250885a9-f7fd-40c1-a3fd-fe6241337751 None None] MFA Rules not processed for user `5a0294ea40914bf290902307a787642d`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:35:36.364099 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-250885a9-f7fd-40c1-a3fd-fe6241337751 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:35:36.402619 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 848/1678] 10.222.0.30 () {58 vars in 958 bytes} [Tue Sep 6 04:35:36 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 71 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 04:35:37.787010 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc6703e-09d4-437d-8355-c5e81ad7dc2a None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:35:37.787528 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc6703e-09d4-437d-8355-c5e81ad7dc2a None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:35:37.787638 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc6703e-09d4-437d-8355-c5e81ad7dc2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:35:37.846391 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-1dc6703e-09d4-437d-8355-c5e81ad7dc2a None None] MFA Rules not processed for user `1ca147cfc97149dd87e533123a262cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:35:37.862045 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-1dc6703e-09d4-437d-8355-c5e81ad7dc2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:35:37.929359 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 831/1679] 10.222.0.30 () {58 vars in 958 bytes} [Tue Sep 6 04:35:37 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 04:35:41.222506 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10aaad9a-e2f2-4344-87ff-d91e8df5abb4 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:35:41.222992 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10aaad9a-e2f2-4344-87ff-d91e8df5abb4 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:35:41.223075 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10aaad9a-e2f2-4344-87ff-d91e8df5abb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:35:41.330104 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-10aaad9a-e2f2-4344-87ff-d91e8df5abb4 None None] MFA Rules not processed for user `da5fed17101a4935bcc38b72caf9d263`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:35:41.331485 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-10aaad9a-e2f2-4344-87ff-d91e8df5abb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:35:41.420648 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:35:41.504781 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 849/1680] 10.222.0.30 () {58 vars in 1009 bytes} [Tue Sep 6 04:35:41 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 283 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 04:35:41.601504 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc6703e-09d4-437d-8355-c5e81ad7dc2a None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:35:41.653665 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:35:41.779916 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc6703e-09d4-437d-8355-c5e81ad7dc2a None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:35:41.786113 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a19ac7-c294-4a54-8d53-71b9de15bb84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:35:41.790056 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a19ac7-c294-4a54-8d53-71b9de15bb84 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:35:41.790701 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a19ac7-c294-4a54-8d53-71b9de15bb84 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:35:41.791325 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a19ac7-c294-4a54-8d53-71b9de15bb84 admin admin] PATH_INFO: `/v3/users/322f6308959a453f8178e405634a116e` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:35:41.794491 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a19ac7-c294-4a54-8d53-71b9de15bb84 admin admin] RBAC: Authorizing `identity:delete_user(user_id=322f6308959a453f8178e405634a116e)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:35:41.797136 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a19ac7-c294-4a54-8d53-71b9de15bb84 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:35:42.073785 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10aaad9a-e2f2-4344-87ff-d91e8df5abb4 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:35:42.075462 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-10aaad9a-e2f2-4344-87ff-d91e8df5abb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:35:42.149121 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-c0a19ac7-c294-4a54-8d53-71b9de15bb84 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '322f6308959a453f8178e405634a116e'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:35:42.248194 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-c0a19ac7-c294-4a54-8d53-71b9de15bb84 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '322f6308959a453f8178e405634a116e'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:35:42.248980 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-c0a19ac7-c294-4a54-8d53-71b9de15bb84 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '322f6308959a453f8178e405634a116e'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:35:42.257753 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-c0a19ac7-c294-4a54-8d53-71b9de15bb84 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '322f6308959a453f8178e405634a116e'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:35:42.295996 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-c0a19ac7-c294-4a54-8d53-71b9de15bb84 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '322f6308959a453f8178e405634a116e'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:35:42.301056 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10aaad9a-e2f2-4344-87ff-d91e8df5abb4 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:35:42.313042 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8375ab48-ac21-4a44-bef7-59216ee73b99 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a676ec7f92a144edbeca8e6029ba9ee6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b707733ce65c42edbc1fd95ad0a1c06c', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:35:42.313551 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 832/1681] 10.222.0.30 () {58 vars in 1224 bytes} [Tue Sep 6 04:35:41 2022] DELETE /identity/v3/users/322f6308959a453f8178e405634a116e => generated 0 bytes in 713 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 04:35:42.313886 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8375ab48-ac21-4a44-bef7-59216ee73b99 service swift] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:35:42.314188 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8375ab48-ac21-4a44-bef7-59216ee73b99 service swift] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:35:42.314496 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8375ab48-ac21-4a44-bef7-59216ee73b99 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:35:42.319456 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe63fbb-82f8-4bd9-bd36-6e78d2abcc6d None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:35:42.322159 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe63fbb-82f8-4bd9-bd36-6e78d2abcc6d None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:35:42.322159 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe63fbb-82f8-4bd9-bd36-6e78d2abcc6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:35:42.322854 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-8375ab48-ac21-4a44-bef7-59216ee73b99 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:35:42.356572 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8375ab48-ac21-4a44-bef7-59216ee73b99 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:35:42.359483 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8375ab48-ac21-4a44-bef7-59216ee73b99 service swift] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:35:42.390123 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-ffe63fbb-82f8-4bd9-bd36-6e78d2abcc6d None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:35:42.394228 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-ffe63fbb-82f8-4bd9-bd36-6e78d2abcc6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:35:42.404262 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 850/1682] 10.222.0.30 () {60 vars in 1384 bytes} [Tue Sep 6 04:35:42 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 06 04:35:42.470272 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c5a780-1d8a-4906-98c2-af5bd8f74acd None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:35:42.470957 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c5a780-1d8a-4906-98c2-af5bd8f74acd None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:35:42.471409 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c5a780-1d8a-4906-98c2-af5bd8f74acd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:35:42.545467 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-65c5a780-1d8a-4906-98c2-af5bd8f74acd None None] MFA Rules not processed for user `90492440f6a6471e8392361c098428aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:35:42.552500 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-65c5a780-1d8a-4906-98c2-af5bd8f74acd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:35:42.574114 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 833/1683] 10.222.0.30 () {58 vars in 958 bytes} [Tue Sep 6 04:35:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 256 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 04:35:42.585111 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe63fbb-82f8-4bd9-bd36-6e78d2abcc6d None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:35:42.587652 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-ffe63fbb-82f8-4bd9-bd36-6e78d2abcc6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:35:42.744890 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:35:42.786163 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe63fbb-82f8-4bd9-bd36-6e78d2abcc6d None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:35:42.791021 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb938813-43c8-433b-bc4b-d6e0956135d2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90492440f6a6471e8392361c098428aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b707733ce65c42edbc1fd95ad0a1c06c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:35:42.792410 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb938813-43c8-433b-bc4b-d6e0956135d2 service neutron] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:35:42.792518 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb938813-43c8-433b-bc4b-d6e0956135d2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:35:42.792796 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb938813-43c8-433b-bc4b-d6e0956135d2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:35:42.817829 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb938813-43c8-433b-bc4b-d6e0956135d2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:35:42.817829 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb938813-43c8-433b-bc4b-d6e0956135d2 service neutron] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:35:42.829124 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 851/1684] 10.222.0.46 () {58 vars in 1015 bytes} [Tue Sep 6 04:35:42 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 360 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 04:35:42.868024 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 834/1685] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Sep 6 04:35:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 284 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 04:35:42.959445 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c5a780-1d8a-4906-98c2-af5bd8f74acd None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:35:42.971670 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c5a780-1d8a-4906-98c2-af5bd8f74acd None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:35:43.022755 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e493b150-60e5-4b50-b27a-6b6197f9cc1e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90492440f6a6471e8392361c098428aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b707733ce65c42edbc1fd95ad0a1c06c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:35:43.022755 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e493b150-60e5-4b50-b27a-6b6197f9cc1e service neutron] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:35:43.022755 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e493b150-60e5-4b50-b27a-6b6197f9cc1e service neutron] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:35:43.022755 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e493b150-60e5-4b50-b27a-6b6197f9cc1e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:35:43.034078 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e493b150-60e5-4b50-b27a-6b6197f9cc1e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:35:43.047000 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e493b150-60e5-4b50-b27a-6b6197f9cc1e service neutron] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:35:43.077711 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 852/1686] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Sep 6 04:35:42 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 04:35:43.841288 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb938813-43c8-433b-bc4b-d6e0956135d2 service neutron] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:35:43.841288 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-fb938813-43c8-433b-bc4b-d6e0956135d2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:35:43.967874 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb938813-43c8-433b-bc4b-d6e0956135d2 service neutron] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:35:43.984908 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5572111f-7367-4c95-9c8a-69baeee1c902 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd09b5ab675634a2eb4b03755743f7d0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b707733ce65c42edbc1fd95ad0a1c06c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:35:43.986241 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5572111f-7367-4c95-9c8a-69baeee1c902 service cinder] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:35:43.988606 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5572111f-7367-4c95-9c8a-69baeee1c902 service cinder] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:35:43.990239 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5572111f-7367-4c95-9c8a-69baeee1c902 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:35:43.996001 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-5572111f-7367-4c95-9c8a-69baeee1c902 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:35:44.278392 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5572111f-7367-4c95-9c8a-69baeee1c902 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:35:44.288568 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5572111f-7367-4c95-9c8a-69baeee1c902 service cinder] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:35:44.322528 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 835/1687] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Sep 6 04:35:43 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 487 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 04:35:45.783538 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e493b150-60e5-4b50-b27a-6b6197f9cc1e service neutron] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:35:45.785339 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-e493b150-60e5-4b50-b27a-6b6197f9cc1e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:35:45.832298 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e493b150-60e5-4b50-b27a-6b6197f9cc1e service neutron] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:35:45.872371 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fddef981-8c17-442e-93b4-34627af18198 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:35:45.872371 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddef981-8c17-442e-93b4-34627af18198 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:35:45.872371 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddef981-8c17-442e-93b4-34627af18198 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:35:45.872371 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddef981-8c17-442e-93b4-34627af18198 admin admin] PATH_INFO: `/v3/users/fe78cd0f722440f2a42d71d2df164ebe` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:35:45.873561 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fddef981-8c17-442e-93b4-34627af18198 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fe78cd0f722440f2a42d71d2df164ebe)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:35:45.873561 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: WARNING py.warnings [None req-fddef981-8c17-442e-93b4-34627af18198 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 06 04:35:45.873561 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: warnings.warn(msg) Sep 06 04:35:45.873561 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: Sep 06 04:35:45.873561 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fddef981-8c17-442e-93b4-34627af18198 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:35:46.128620 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5572111f-7367-4c95-9c8a-69baeee1c902 service cinder] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:35:46.131025 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-5572111f-7367-4c95-9c8a-69baeee1c902 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:35:46.139327 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:35:46.173643 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5572111f-7367-4c95-9c8a-69baeee1c902 service cinder] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:35:46.185514 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-fddef981-8c17-442e-93b4-34627af18198 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fe78cd0f722440f2a42d71d2df164ebe'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:35:46.186928 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98fb089-c0e9-4583-8e36-afb5e3575129 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:35:46.187708 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98fb089-c0e9-4583-8e36-afb5e3575129 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:35:46.188168 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98fb089-c0e9-4583-8e36-afb5e3575129 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:35:46.188683 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98fb089-c0e9-4583-8e36-afb5e3575129 admin admin] PATH_INFO: `/v3/projects/c0fa3c60782e4830b8c02ab8770e8d60` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:35:46.190058 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d98fb089-c0e9-4583-8e36-afb5e3575129 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c0fa3c60782e4830b8c02ab8770e8d60)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:35:46.192220 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: WARNING py.warnings [None req-d98fb089-c0e9-4583-8e36-afb5e3575129 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 06 04:35:46.192220 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: warnings.warn(msg) Sep 06 04:35:46.192220 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: Sep 06 04:35:46.193775 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d98fb089-c0e9-4583-8e36-afb5e3575129 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:35:46.282761 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-fddef981-8c17-442e-93b4-34627af18198 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fe78cd0f722440f2a42d71d2df164ebe'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:35:46.283606 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-fddef981-8c17-442e-93b4-34627af18198 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fe78cd0f722440f2a42d71d2df164ebe'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:35:46.325809 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-fddef981-8c17-442e-93b4-34627af18198 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fe78cd0f722440f2a42d71d2df164ebe'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:35:46.344708 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-fddef981-8c17-442e-93b4-34627af18198 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fe78cd0f722440f2a42d71d2df164ebe'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:35:46.356657 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 853/1688] 10.222.0.30 () {58 vars in 1224 bytes} [Tue Sep 6 04:35:45 2022] DELETE /identity/v3/users/fe78cd0f722440f2a42d71d2df164ebe => generated 0 bytes in 574 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 04:35:46.378205 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fddef981-8c17-442e-93b4-34627af18198 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:35:46.378602 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-fddef981-8c17-442e-93b4-34627af18198 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:35:46.427786 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-d98fb089-c0e9-4583-8e36-afb5e3575129 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c0fa3c60782e4830b8c02ab8770e8d60', 'request_id': 'req-d98fb089-c0e9-4583-8e36-afb5e3575129'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:35:46.549969 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fddef981-8c17-442e-93b4-34627af18198 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:35:46.556971 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa074053-18f4-4e24-bb6c-43e9db78e795 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:35:46.560492 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa074053-18f4-4e24-bb6c-43e9db78e795 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:35:46.561003 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa074053-18f4-4e24-bb6c-43e9db78e795 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:35:46.561412 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa074053-18f4-4e24-bb6c-43e9db78e795 admin admin] PATH_INFO: `/v3/users/5dcd5ff1bdc342539fb4fa154878bf3d` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:35:46.563166 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa074053-18f4-4e24-bb6c-43e9db78e795 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5dcd5ff1bdc342539fb4fa154878bf3d)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:35:46.565797 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa074053-18f4-4e24-bb6c-43e9db78e795 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:35:46.594975 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-d98fb089-c0e9-4583-8e36-afb5e3575129 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c0fa3c60782e4830b8c02ab8770e8d60', 'request_id': 'req-d98fb089-c0e9-4583-8e36-afb5e3575129'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:35:46.645203 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-d98fb089-c0e9-4583-8e36-afb5e3575129 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c0fa3c60782e4830b8c02ab8770e8d60', 'request_id': 'req-d98fb089-c0e9-4583-8e36-afb5e3575129'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:35:46.684339 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-d98fb089-c0e9-4583-8e36-afb5e3575129 admin admin] The token cache is being invalidate because project c0fa3c60782e4830b8c02ab8770e8d60 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81697) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 04:35:46.689656 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-d98fb089-c0e9-4583-8e36-afb5e3575129 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:35:46.689656 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-d98fb089-c0e9-4583-8e36-afb5e3575129 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:35:46.689656 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 836/1689] 10.222.0.30 () {58 vars in 1233 bytes} [Tue Sep 6 04:35:46 2022] DELETE /identity/v3/projects/c0fa3c60782e4830b8c02ab8770e8d60 => generated 0 bytes in 560 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 04:35:46.689656 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d71dcf1-1bca-480f-bbbe-a7adcec29aa2 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:35:46.689656 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d71dcf1-1bca-480f-bbbe-a7adcec29aa2 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:35:46.689656 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d71dcf1-1bca-480f-bbbe-a7adcec29aa2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:35:46.725555 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-1d71dcf1-1bca-480f-bbbe-a7adcec29aa2 None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:35:46.727248 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-1d71dcf1-1bca-480f-bbbe-a7adcec29aa2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:35:46.789635 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 837/1690] 10.222.0.30 () {58 vars in 958 bytes} [Tue Sep 6 04:35:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 04:35:46.789635 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e24ca3-6f9d-47dc-9d46-3bd3ef4e81f1 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:35:46.789635 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e24ca3-6f9d-47dc-9d46-3bd3ef4e81f1 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:35:46.789635 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e24ca3-6f9d-47dc-9d46-3bd3ef4e81f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:35:46.801191 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-85e24ca3-6f9d-47dc-9d46-3bd3ef4e81f1 None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:35:46.803389 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-85e24ca3-6f9d-47dc-9d46-3bd3ef4e81f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:35:46.823972 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 838/1691] 10.222.0.30 () {58 vars in 958 bytes} [Tue Sep 6 04:35:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 04:35:46.831843 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edde648a-a056-4e4f-91f1-55effbb3ce58 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:35:46.832724 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edde648a-a056-4e4f-91f1-55effbb3ce58 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:35:46.839675 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edde648a-a056-4e4f-91f1-55effbb3ce58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:35:46.919847 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:35:46.920153 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-fa074053-18f4-4e24-bb6c-43e9db78e795 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5dcd5ff1bdc342539fb4fa154878bf3d'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:35:46.951625 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-edde648a-a056-4e4f-91f1-55effbb3ce58 None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:35:46.954986 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-edde648a-a056-4e4f-91f1-55effbb3ce58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:35:47.005231 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 839/1692] 10.222.0.30 () {58 vars in 958 bytes} [Tue Sep 6 04:35:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 04:35:47.006838 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edde648a-a056-4e4f-91f1-55effbb3ce58 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:35:47.042274 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-fa074053-18f4-4e24-bb6c-43e9db78e795 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5dcd5ff1bdc342539fb4fa154878bf3d'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:35:47.043928 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-fa074053-18f4-4e24-bb6c-43e9db78e795 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5dcd5ff1bdc342539fb4fa154878bf3d'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:35:47.073436 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edde648a-a056-4e4f-91f1-55effbb3ce58 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:35:47.097577 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-fa074053-18f4-4e24-bb6c-43e9db78e795 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5dcd5ff1bdc342539fb4fa154878bf3d'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:35:47.106624 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2d4db1-c521-42ff-9024-80c49c1bcad3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:35:47.107437 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2d4db1-c521-42ff-9024-80c49c1bcad3 admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:35:47.107729 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2d4db1-c521-42ff-9024-80c49c1bcad3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:35:47.108532 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2d4db1-c521-42ff-9024-80c49c1bcad3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:35:47.111068 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a2d4db1-c521-42ff-9024-80c49c1bcad3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:35:47.114018 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a2d4db1-c521-42ff-9024-80c49c1bcad3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:35:47.115908 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a2d4db1-c521-42ff-9024-80c49c1bcad3 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:35:47.130589 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-fa074053-18f4-4e24-bb6c-43e9db78e795 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5dcd5ff1bdc342539fb4fa154878bf3d'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:35:47.172796 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 840/1693] 10.222.0.30 () {58 vars in 1153 bytes} [Tue Sep 6 04:35:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:35:47.176669 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c75f5a-9c0a-4c22-ad67-cc83a0b198f9 None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:35:47.177612 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c75f5a-9c0a-4c22-ad67-cc83a0b198f9 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:35:47.177991 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c75f5a-9c0a-4c22-ad67-cc83a0b198f9 None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:35:47.186965 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 841/1694] 10.222.0.30 () {54 vars in 921 bytes} [Tue Sep 6 04:35:47 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 04:35:47.186965 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c75f5a-9c0a-4c22-ad67-cc83a0b198f9 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:35:47.187443 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 854/1695] 10.222.0.30 () {58 vars in 1224 bytes} [Tue Sep 6 04:35:46 2022] DELETE /identity/v3/users/5dcd5ff1bdc342539fb4fa154878bf3d => generated 0 bytes in 812 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 04:35:47.191238 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa074053-18f4-4e24-bb6c-43e9db78e795 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:35:47.191499 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-80c75f5a-9c0a-4c22-ad67-cc83a0b198f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:35:47.197335 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-fa074053-18f4-4e24-bb6c-43e9db78e795 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:35:47.282576 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:35:47.345770 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa074053-18f4-4e24-bb6c-43e9db78e795 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:35:47.355510 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91a1e65-61d6-4c95-99dc-8e61a0142e21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:35:47.356294 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91a1e65-61d6-4c95-99dc-8e61a0142e21 admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:35:47.356422 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91a1e65-61d6-4c95-99dc-8e61a0142e21 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:35:47.356795 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91a1e65-61d6-4c95-99dc-8e61a0142e21 admin admin] PATH_INFO: `/v3/projects` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:35:47.358613 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e91a1e65-61d6-4c95-99dc-8e61a0142e21 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:35:47.360685 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e91a1e65-61d6-4c95-99dc-8e61a0142e21 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:35:47.389780 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c75f5a-9c0a-4c22-ad67-cc83a0b198f9 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:35:47.414364 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1a7b59-1262-45db-9e1e-8fc91bb55f5b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90492440f6a6471e8392361c098428aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b707733ce65c42edbc1fd95ad0a1c06c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:35:47.415265 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1a7b59-1262-45db-9e1e-8fc91bb55f5b service neutron] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:35:47.415559 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1a7b59-1262-45db-9e1e-8fc91bb55f5b service neutron] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:35:47.415897 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1a7b59-1262-45db-9e1e-8fc91bb55f5b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:35:47.420506 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-6c1a7b59-1262-45db-9e1e-8fc91bb55f5b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:35:47.444759 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1a7b59-1262-45db-9e1e-8fc91bb55f5b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:35:47.446608 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1a7b59-1262-45db-9e1e-8fc91bb55f5b service neutron] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:35:47.472900 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 842/1696] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Sep 6 04:35:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 289 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 04:35:47.475083 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959403cf-82dc-4c75-966f-42e84961f81d None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:35:47.475083 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959403cf-82dc-4c75-966f-42e84961f81d None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:35:47.475083 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959403cf-82dc-4c75-966f-42e84961f81d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:35:47.496917 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 855/1697] 10.222.0.30 () {60 vars in 1153 bytes} [Tue Sep 6 04:35:47 2022] POST /identity/v3/projects => generated 365 bytes in 310 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 04:35:47.511336 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6b5e75-4d43-4871-a55e-e19eea390afc None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:35:47.513085 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6b5e75-4d43-4871-a55e-e19eea390afc None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:35:47.513615 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6b5e75-4d43-4871-a55e-e19eea390afc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:35:47.553367 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-7c6b5e75-4d43-4871-a55e-e19eea390afc None None] MFA Rules not processed for user `9db138b77afa4b068244b8ef715404fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:35:47.555228 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-7c6b5e75-4d43-4871-a55e-e19eea390afc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:35:47.569016 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-959403cf-82dc-4c75-966f-42e84961f81d None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:35:47.575218 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-959403cf-82dc-4c75-966f-42e84961f81d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:35:47.739154 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 856/1698] 10.222.0.30 () {58 vars in 1018 bytes} [Tue Sep 6 04:35:47 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 242 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 04:35:47.740202 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6b5e75-4d43-4871-a55e-e19eea390afc None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:35:47.744191 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 843/1699] 10.222.0.30 () {58 vars in 958 bytes} [Tue Sep 6 04:35:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 271 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 04:35:47.766329 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6b5e75-4d43-4871-a55e-e19eea390afc None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:35:47.772784 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4dc6e42-b1b2-4f00-ad99-38c6223fbec4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:35:47.774544 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4dc6e42-b1b2-4f00-ad99-38c6223fbec4 admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:35:47.775241 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4dc6e42-b1b2-4f00-ad99-38c6223fbec4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:35:47.775883 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4dc6e42-b1b2-4f00-ad99-38c6223fbec4 admin admin] PATH_INFO: `/v3/users` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:35:47.777541 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4dc6e42-b1b2-4f00-ad99-38c6223fbec4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:35:47.780862 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4dc6e42-b1b2-4f00-ad99-38c6223fbec4 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:35:47.837430 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959403cf-82dc-4c75-966f-42e84961f81d None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:35:47.840397 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-959403cf-82dc-4c75-966f-42e84961f81d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:35:47.918552 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 857/1700] 10.222.0.30 () {60 vars in 1144 bytes} [Tue Sep 6 04:35:47 2022] POST /identity/v3/users => generated 283 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 04:35:47.923332 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4dc6e42-b1b2-4f00-ad99-38c6223fbec4 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:35:47.958621 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4dc6e42-b1b2-4f00-ad99-38c6223fbec4 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:35:47.968841 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e118ded-a949-4cb7-a596-80f832507c59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:35:47.969636 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e118ded-a949-4cb7-a596-80f832507c59 admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:35:47.969923 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e118ded-a949-4cb7-a596-80f832507c59 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:35:47.970448 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e118ded-a949-4cb7-a596-80f832507c59 admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:35:47.971746 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e118ded-a949-4cb7-a596-80f832507c59 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:35:47.974354 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e118ded-a949-4cb7-a596-80f832507c59 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:35:47.997642 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959403cf-82dc-4c75-966f-42e84961f81d None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:35:48.003481 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 858/1701] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Sep 6 04:35:47 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 04:35:48.005868 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26641573-74a3-4bf9-beb6-f5e8fae3a95b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90492440f6a6471e8392361c098428aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b707733ce65c42edbc1fd95ad0a1c06c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:35:48.007926 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e118ded-a949-4cb7-a596-80f832507c59 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:35:48.011060 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26641573-74a3-4bf9-beb6-f5e8fae3a95b service neutron] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:35:48.014018 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26641573-74a3-4bf9-beb6-f5e8fae3a95b service neutron] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:35:48.014323 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26641573-74a3-4bf9-beb6-f5e8fae3a95b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:35:48.038580 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e118ded-a949-4cb7-a596-80f832507c59 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:35:48.044730 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26641573-74a3-4bf9-beb6-f5e8fae3a95b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:35:48.046805 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26641573-74a3-4bf9-beb6-f5e8fae3a95b service neutron] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:35:48.054487 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37042dc9-6a08-4efa-9539-16d40d97cf5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:35:48.055921 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37042dc9-6a08-4efa-9539-16d40d97cf5f admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:35:48.055921 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37042dc9-6a08-4efa-9539-16d40d97cf5f admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:35:48.055921 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37042dc9-6a08-4efa-9539-16d40d97cf5f admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:35:48.056598 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37042dc9-6a08-4efa-9539-16d40d97cf5f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:35:48.063132 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37042dc9-6a08-4efa-9539-16d40d97cf5f admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:35:48.080195 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 844/1702] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Sep 6 04:35:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 243 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 04:35:48.085538 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 859/1703] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Sep 6 04:35:48 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 04:35:48.090811 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26641573-74a3-4bf9-beb6-f5e8fae3a95b service neutron] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:35:48.104552 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26641573-74a3-4bf9-beb6-f5e8fae3a95b service neutron] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:35:48.110864 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33776f8c-d27e-453c-a1e3-e68d0d83aa00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:35:48.111743 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33776f8c-d27e-453c-a1e3-e68d0d83aa00 admin admin] REQUEST_METHOD: `PUT` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:35:48.112227 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33776f8c-d27e-453c-a1e3-e68d0d83aa00 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:35:48.112773 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33776f8c-d27e-453c-a1e3-e68d0d83aa00 admin admin] PATH_INFO: `/v3/projects/6d29ea41356640f9871d15a5dfe8a42d/users/eaa8037d5b674905b7a31cf11dd5791d/roles/29607b2de39f40d68bb6ceef64b8a732` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:35:48.136868 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33776f8c-d27e-453c-a1e3-e68d0d83aa00 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6d29ea41356640f9871d15a5dfe8a42d, user_id=eaa8037d5b674905b7a31cf11dd5791d, role_id=29607b2de39f40d68bb6ceef64b8a732)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:35:48.143654 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33776f8c-d27e-453c-a1e3-e68d0d83aa00 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:35:48.360875 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37042dc9-6a08-4efa-9539-16d40d97cf5f admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:35:48.369398 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-37042dc9-6a08-4efa-9539-16d40d97cf5f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:35:48.426987 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37042dc9-6a08-4efa-9539-16d40d97cf5f admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:35:48.445051 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d8e6e5-6857-42b8-acc5-c63633d62f23 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9db138b77afa4b068244b8ef715404fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b707733ce65c42edbc1fd95ad0a1c06c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:35:48.447222 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d8e6e5-6857-42b8-acc5-c63633d62f23 service nova] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:35:48.455619 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d8e6e5-6857-42b8-acc5-c63633d62f23 service nova] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:35:48.457094 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d8e6e5-6857-42b8-acc5-c63633d62f23 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:35:48.483512 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d8e6e5-6857-42b8-acc5-c63633d62f23 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:35:48.489664 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d8e6e5-6857-42b8-acc5-c63633d62f23 service nova] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:35:48.522973 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 860/1704] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Sep 6 04:35:48 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 04:35:48.629054 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 845/1705] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Sep 6 04:35:48 2022] PUT /identity/v3/projects/6d29ea41356640f9871d15a5dfe8a42d/users/eaa8037d5b674905b7a31cf11dd5791d/roles/29607b2de39f40d68bb6ceef64b8a732 => generated 0 bytes in 540 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 04:35:48.642959 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adaab888-d589-4096-8f5d-dc1c9cec1234 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:35:48.643252 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adaab888-d589-4096-8f5d-dc1c9cec1234 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:35:48.643534 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adaab888-d589-4096-8f5d-dc1c9cec1234 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:35:48.683448 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-adaab888-d589-4096-8f5d-dc1c9cec1234 None None] MFA Rules not processed for user `eaa8037d5b674905b7a31cf11dd5791d`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:35:48.684752 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-adaab888-d589-4096-8f5d-dc1c9cec1234 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:35:48.840524 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 861/1706] 10.222.0.30 () {58 vars in 958 bytes} [Tue Sep 6 04:35:48 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 04:35:48.847304 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33776f8c-d27e-453c-a1e3-e68d0d83aa00 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:35:49.015111 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:35:49.078139 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33776f8c-d27e-453c-a1e3-e68d0d83aa00 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:35:49.084185 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c60984-cdaa-4f53-902e-791541e715a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:35:49.086077 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c60984-cdaa-4f53-902e-791541e715a4 admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:35:49.086332 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c60984-cdaa-4f53-902e-791541e715a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:35:49.086616 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c60984-cdaa-4f53-902e-791541e715a4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:35:49.087609 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c60984-cdaa-4f53-902e-791541e715a4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:35:49.095789 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c60984-cdaa-4f53-902e-791541e715a4 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:35:49.097543 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f847e32-4244-4458-8704-df3ae0e46c16 None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:35:49.098558 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f847e32-4244-4458-8704-df3ae0e46c16 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:35:49.099040 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f847e32-4244-4458-8704-df3ae0e46c16 None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:35:49.101091 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 862/1707] 10.222.0.30 () {54 vars in 921 bytes} [Tue Sep 6 04:35:49 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 04:35:49.113027 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb27687-9203-4c01-a386-f78265622d26 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:35:49.113668 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb27687-9203-4c01-a386-f78265622d26 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:35:49.114160 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb27687-9203-4c01-a386-f78265622d26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:35:49.164838 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-6fb27687-9203-4c01-a386-f78265622d26 None None] MFA Rules not processed for user `9db138b77afa4b068244b8ef715404fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:35:49.170794 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-6fb27687-9203-4c01-a386-f78265622d26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:35:49.228426 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 846/1708] 10.222.0.30 () {60 vars in 1153 bytes} [Tue Sep 6 04:35:48 2022] POST /identity/v3/projects => generated 363 bytes in 381 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 04:35:49.231295 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c60984-cdaa-4f53-902e-791541e715a4 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:35:49.466252 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c60984-cdaa-4f53-902e-791541e715a4 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:35:49.473718 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 863/1709] 10.222.0.30 () {58 vars in 1018 bytes} [Tue Sep 6 04:35:49 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 361 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 04:35:49.475699 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a074aeb2-795e-4728-956d-b71660d0b9a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:35:49.476343 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a074aeb2-795e-4728-956d-b71660d0b9a1 admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:35:49.476624 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a074aeb2-795e-4728-956d-b71660d0b9a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:35:49.476897 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a074aeb2-795e-4728-956d-b71660d0b9a1 admin admin] PATH_INFO: `/v3/users` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:35:49.477594 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a074aeb2-795e-4728-956d-b71660d0b9a1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:35:49.480502 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a074aeb2-795e-4728-956d-b71660d0b9a1 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:35:49.608540 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 847/1710] 10.222.0.30 () {60 vars in 1144 bytes} [Tue Sep 6 04:35:49 2022] POST /identity/v3/users => generated 288 bytes in 378 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 04:35:49.616192 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb27687-9203-4c01-a386-f78265622d26 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:35:49.635216 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb27687-9203-4c01-a386-f78265622d26 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:35:49.652132 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2979c7-f214-4cbe-98bc-0a3691cc5a82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:35:49.653027 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2979c7-f214-4cbe-98bc-0a3691cc5a82 admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:35:49.653553 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2979c7-f214-4cbe-98bc-0a3691cc5a82 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:35:49.654137 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2979c7-f214-4cbe-98bc-0a3691cc5a82 admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:35:49.655047 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2979c7-f214-4cbe-98bc-0a3691cc5a82 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:35:49.657139 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2979c7-f214-4cbe-98bc-0a3691cc5a82 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:35:49.690353 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 864/1711] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Sep 6 04:35:49 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 04:35:49.694223 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a074aeb2-795e-4728-956d-b71660d0b9a1 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:35:49.711785 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a074aeb2-795e-4728-956d-b71660d0b9a1 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:35:49.723918 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6193567c-94e6-4299-a8fc-eca8a9fadc49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:35:49.725930 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6193567c-94e6-4299-a8fc-eca8a9fadc49 admin admin] REQUEST_METHOD: `PUT` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:35:49.727225 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6193567c-94e6-4299-a8fc-eca8a9fadc49 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:35:49.727225 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6193567c-94e6-4299-a8fc-eca8a9fadc49 admin admin] PATH_INFO: `/v3/projects/1e6259212e5648f4b668a7d84767bacd/users/863d0ca50601424fac9c22be0caba6ad/roles/ea4c814713b149108e488de855b025e4` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:35:49.759901 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6193567c-94e6-4299-a8fc-eca8a9fadc49 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1e6259212e5648f4b668a7d84767bacd, user_id=863d0ca50601424fac9c22be0caba6ad, role_id=ea4c814713b149108e488de855b025e4)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:35:49.766484 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6193567c-94e6-4299-a8fc-eca8a9fadc49 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:35:49.872779 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:35:49.886563 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 848/1712] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Sep 6 04:35:49 2022] PUT /identity/v3/projects/1e6259212e5648f4b668a7d84767bacd/users/863d0ca50601424fac9c22be0caba6ad/roles/ea4c814713b149108e488de855b025e4 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 04:35:49.886845 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2979c7-f214-4cbe-98bc-0a3691cc5a82 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:35:49.906400 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2979c7-f214-4cbe-98bc-0a3691cc5a82 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:35:50.145859 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6193567c-94e6-4299-a8fc-eca8a9fadc49 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:35:50.147314 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0485ca-d7bf-4541-9b0d-4469974b7227 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:35:50.147314 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0485ca-d7bf-4541-9b0d-4469974b7227 admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:35:50.147314 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0485ca-d7bf-4541-9b0d-4469974b7227 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:35:50.147314 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0485ca-d7bf-4541-9b0d-4469974b7227 admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:35:50.148256 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0485ca-d7bf-4541-9b0d-4469974b7227 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:35:50.148256 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0485ca-d7bf-4541-9b0d-4469974b7227 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:35:50.148256 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 865/1713] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Sep 6 04:35:49 2022] GET /identity/v3/roles => generated 1420 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 04:35:50.167478 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0485ca-d7bf-4541-9b0d-4469974b7227 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:35:50.190372 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0485ca-d7bf-4541-9b0d-4469974b7227 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:35:50.221806 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6193567c-94e6-4299-a8fc-eca8a9fadc49 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:35:50.226170 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a90c2c6-e161-4f9a-b94a-f8d939704b6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:35:50.226170 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a90c2c6-e161-4f9a-b94a-f8d939704b6d admin admin] REQUEST_METHOD: `PUT` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:35:50.226170 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a90c2c6-e161-4f9a-b94a-f8d939704b6d admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:35:50.226460 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a90c2c6-e161-4f9a-b94a-f8d939704b6d admin admin] PATH_INFO: `/v3/projects/1e6259212e5648f4b668a7d84767bacd/users/863d0ca50601424fac9c22be0caba6ad/roles/ea4c814713b149108e488de855b025e4` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:35:50.235010 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a90c2c6-e161-4f9a-b94a-f8d939704b6d admin admin] RBAC: Authorizing `identity:create_grant(project_id=1e6259212e5648f4b668a7d84767bacd, user_id=863d0ca50601424fac9c22be0caba6ad, role_id=ea4c814713b149108e488de855b025e4)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:35:50.242637 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd846ac1-2875-49d6-b11a-4b23a9e89a70 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9db138b77afa4b068244b8ef715404fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b707733ce65c42edbc1fd95ad0a1c06c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:35:50.243387 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd846ac1-2875-49d6-b11a-4b23a9e89a70 service nova] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:35:50.243666 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd846ac1-2875-49d6-b11a-4b23a9e89a70 service nova] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:35:50.244043 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd846ac1-2875-49d6-b11a-4b23a9e89a70 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:35:50.244546 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a90c2c6-e161-4f9a-b94a-f8d939704b6d admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:35:50.285659 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd846ac1-2875-49d6-b11a-4b23a9e89a70 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:35:50.293284 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd846ac1-2875-49d6-b11a-4b23a9e89a70 service nova] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:35:50.324220 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 866/1714] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Sep 6 04:35:50 2022] PUT /identity/v3/projects/1e6259212e5648f4b668a7d84767bacd/users/863d0ca50601424fac9c22be0caba6ad/roles/ea4c814713b149108e488de855b025e4 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 04:35:50.367719 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75de41e1-9480-4fe5-a4fa-4b6c31416c1a None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:35:50.369701 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75de41e1-9480-4fe5-a4fa-4b6c31416c1a None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:35:50.372360 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 849/1715] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Sep 6 04:35:50 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 06 04:35:50.380574 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75de41e1-9480-4fe5-a4fa-4b6c31416c1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:35:50.423490 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd846ac1-2875-49d6-b11a-4b23a9e89a70 service nova] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:35:50.425160 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-bd846ac1-2875-49d6-b11a-4b23a9e89a70 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:35:50.475722 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd846ac1-2875-49d6-b11a-4b23a9e89a70 service nova] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:35:50.563594 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-75de41e1-9480-4fe5-a4fa-4b6c31416c1a None None] MFA Rules not processed for user `863d0ca50601424fac9c22be0caba6ad`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:35:50.565245 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-75de41e1-9480-4fe5-a4fa-4b6c31416c1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:35:50.693473 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7420b4-ce61-42b7-afd2-ff18f850981d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:35:50.694164 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7420b4-ce61-42b7-afd2-ff18f850981d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:35:50.694443 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7420b4-ce61-42b7-afd2-ff18f850981d admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:35:50.695036 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7420b4-ce61-42b7-afd2-ff18f850981d admin admin] PATH_INFO: `/v3/projects/6852f495350f402a9b2280e91d2df088` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:35:50.700037 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7420b4-ce61-42b7-afd2-ff18f850981d admin admin] RBAC: Authorizing `identity:delete_project(project_id=6852f495350f402a9b2280e91d2df088)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:35:50.704564 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7420b4-ce61-42b7-afd2-ff18f850981d admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:35:50.738451 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 867/1716] 10.222.0.30 () {58 vars in 958 bytes} [Tue Sep 6 04:35:50 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 396 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 04:35:50.738451 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75de41e1-9480-4fe5-a4fa-4b6c31416c1a None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:35:50.738451 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-75de41e1-9480-4fe5-a4fa-4b6c31416c1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:35:50.760239 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75de41e1-9480-4fe5-a4fa-4b6c31416c1a None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:35:50.780300 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e2cc94-5458-4f0b-9ac5-9ba07556aef3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:35:50.781815 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e2cc94-5458-4f0b-9ac5-9ba07556aef3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:35:50.782535 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e2cc94-5458-4f0b-9ac5-9ba07556aef3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:35:50.783129 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e2cc94-5458-4f0b-9ac5-9ba07556aef3 admin admin] PATH_INFO: `/v3/projects/a3ab4d7d42b2424a988365042e839456` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:35:50.784955 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e2cc94-5458-4f0b-9ac5-9ba07556aef3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a3ab4d7d42b2424a988365042e839456)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:35:50.789397 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e2cc94-5458-4f0b-9ac5-9ba07556aef3 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:35:50.959632 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-0e7420b4-ce61-42b7-afd2-ff18f850981d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6852f495350f402a9b2280e91d2df088', 'request_id': 'req-0e7420b4-ce61-42b7-afd2-ff18f850981d'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:35:51.006956 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-49e2cc94-5458-4f0b-9ac5-9ba07556aef3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a3ab4d7d42b2424a988365042e839456', 'request_id': 'req-49e2cc94-5458-4f0b-9ac5-9ba07556aef3'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:35:51.029246 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-49e2cc94-5458-4f0b-9ac5-9ba07556aef3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a3ab4d7d42b2424a988365042e839456', 'request_id': 'req-49e2cc94-5458-4f0b-9ac5-9ba07556aef3'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:35:51.340412 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-0e7420b4-ce61-42b7-afd2-ff18f850981d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6852f495350f402a9b2280e91d2df088', 'request_id': 'req-0e7420b4-ce61-42b7-afd2-ff18f850981d'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:35:51.350685 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-0e7420b4-ce61-42b7-afd2-ff18f850981d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6852f495350f402a9b2280e91d2df088', 'request_id': 'req-0e7420b4-ce61-42b7-afd2-ff18f850981d'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:35:51.372982 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-0e7420b4-ce61-42b7-afd2-ff18f850981d admin admin] The token cache is being invalidate because project 6852f495350f402a9b2280e91d2df088 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81697) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 04:35:51.374327 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-0e7420b4-ce61-42b7-afd2-ff18f850981d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:35:51.389217 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-0e7420b4-ce61-42b7-afd2-ff18f850981d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:35:51.393051 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-49e2cc94-5458-4f0b-9ac5-9ba07556aef3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a3ab4d7d42b2424a988365042e839456', 'request_id': 'req-49e2cc94-5458-4f0b-9ac5-9ba07556aef3'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:35:51.398734 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 850/1717] 10.222.0.30 () {58 vars in 1233 bytes} [Tue Sep 6 04:35:50 2022] DELETE /identity/v3/projects/6852f495350f402a9b2280e91d2df088 => generated 0 bytes in 976 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 04:35:51.401787 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7420b4-ce61-42b7-afd2-ff18f850981d admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:35:51.407333 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-0e7420b4-ce61-42b7-afd2-ff18f850981d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:35:51.415105 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-49e2cc94-5458-4f0b-9ac5-9ba07556aef3 admin admin] The token cache is being invalidate because project a3ab4d7d42b2424a988365042e839456 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81698) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 04:35:51.416315 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-49e2cc94-5458-4f0b-9ac5-9ba07556aef3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:35:51.420297 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-49e2cc94-5458-4f0b-9ac5-9ba07556aef3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:35:51.436442 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 868/1718] 10.222.0.30 () {58 vars in 1233 bytes} [Tue Sep 6 04:35:50 2022] DELETE /identity/v3/projects/a3ab4d7d42b2424a988365042e839456 => generated 0 bytes in 697 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 04:35:51.436442 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e2cc94-5458-4f0b-9ac5-9ba07556aef3 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:35:51.446088 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-49e2cc94-5458-4f0b-9ac5-9ba07556aef3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:35:51.581843 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7420b4-ce61-42b7-afd2-ff18f850981d admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:35:51.746702 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e2cc94-5458-4f0b-9ac5-9ba07556aef3 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:35:51.746864 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b87d05c-8bed-47f7-b237-51d5340e3642 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd09b5ab675634a2eb4b03755743f7d0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b707733ce65c42edbc1fd95ad0a1c06c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:35:51.746864 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b87d05c-8bed-47f7-b237-51d5340e3642 service cinder] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:35:51.746864 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b87d05c-8bed-47f7-b237-51d5340e3642 service cinder] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:35:51.746864 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b87d05c-8bed-47f7-b237-51d5340e3642 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:35:51.747613 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-9b87d05c-8bed-47f7-b237-51d5340e3642 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:35:51.747689 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e77d33a-73e1-436a-a23f-31f2837e3f7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:35:51.747689 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e77d33a-73e1-436a-a23f-31f2837e3f7b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:35:51.747689 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e77d33a-73e1-436a-a23f-31f2837e3f7b admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:35:51.747689 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e77d33a-73e1-436a-a23f-31f2837e3f7b admin admin] PATH_INFO: `/v3/users/c211020f686f4c8ca5874f00c853b963` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:35:51.748016 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e77d33a-73e1-436a-a23f-31f2837e3f7b admin admin] RBAC: Authorizing `identity:delete_user(user_id=c211020f686f4c8ca5874f00c853b963)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:35:51.748016 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e77d33a-73e1-436a-a23f-31f2837e3f7b admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:35:51.859832 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b87d05c-8bed-47f7-b237-51d5340e3642 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:35:51.862637 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b87d05c-8bed-47f7-b237-51d5340e3642 service cinder] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:35:51.883574 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 851/1719] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Sep 6 04:35:51 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 483 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 04:35:51.883574 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b87d05c-8bed-47f7-b237-51d5340e3642 service cinder] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:35:51.885018 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-9b87d05c-8bed-47f7-b237-51d5340e3642 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:35:51.933766 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b87d05c-8bed-47f7-b237-51d5340e3642 service cinder] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:35:51.951423 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391cea69-2c6f-442a-a617-f8f8063571d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:35:51.952131 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391cea69-2c6f-442a-a617-f8f8063571d0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:35:51.952428 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391cea69-2c6f-442a-a617-f8f8063571d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:35:51.952674 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391cea69-2c6f-442a-a617-f8f8063571d0 admin admin] PATH_INFO: `/v3/users/9df0ff1eb10946e192622a26876d1674` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:35:51.955840 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-391cea69-2c6f-442a-a617-f8f8063571d0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9df0ff1eb10946e192622a26876d1674)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:35:51.957934 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-391cea69-2c6f-442a-a617-f8f8063571d0 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:35:52.230392 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-9e77d33a-73e1-436a-a23f-31f2837e3f7b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c211020f686f4c8ca5874f00c853b963'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:35:52.489932 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-9e77d33a-73e1-436a-a23f-31f2837e3f7b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c211020f686f4c8ca5874f00c853b963'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:35:52.495592 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-9e77d33a-73e1-436a-a23f-31f2837e3f7b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c211020f686f4c8ca5874f00c853b963'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:35:52.511260 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:35:52.513579 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-391cea69-2c6f-442a-a617-f8f8063571d0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9df0ff1eb10946e192622a26876d1674'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:35:52.529115 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-9e77d33a-73e1-436a-a23f-31f2837e3f7b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c211020f686f4c8ca5874f00c853b963'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:35:52.540083 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-9e77d33a-73e1-436a-a23f-31f2837e3f7b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c211020f686f4c8ca5874f00c853b963'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:35:52.549711 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 869/1720] 10.222.0.30 () {58 vars in 1224 bytes} [Tue Sep 6 04:35:51 2022] DELETE /identity/v3/users/c211020f686f4c8ca5874f00c853b963 => generated 0 bytes in 1118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 04:35:52.557091 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e77d33a-73e1-436a-a23f-31f2837e3f7b admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:35:52.559507 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-9e77d33a-73e1-436a-a23f-31f2837e3f7b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:35:52.601217 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-391cea69-2c6f-442a-a617-f8f8063571d0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9df0ff1eb10946e192622a26876d1674'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:35:52.604807 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-391cea69-2c6f-442a-a617-f8f8063571d0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9df0ff1eb10946e192622a26876d1674'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:35:52.615136 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-391cea69-2c6f-442a-a617-f8f8063571d0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9df0ff1eb10946e192622a26876d1674'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:35:52.643310 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-391cea69-2c6f-442a-a617-f8f8063571d0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9df0ff1eb10946e192622a26876d1674'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:35:52.670628 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 852/1721] 10.222.0.30 () {58 vars in 1224 bytes} [Tue Sep 6 04:35:51 2022] DELETE /identity/v3/users/9df0ff1eb10946e192622a26876d1674 => generated 0 bytes in 787 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 04:35:52.726810 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e77d33a-73e1-436a-a23f-31f2837e3f7b admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:35:52.960468 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201db7e1-2041-4e93-ab3e-699cee3cc743 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:35:52.960468 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201db7e1-2041-4e93-ab3e-699cee3cc743 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:35:52.960468 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201db7e1-2041-4e93-ab3e-699cee3cc743 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:35:52.960468 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201db7e1-2041-4e93-ab3e-699cee3cc743 admin admin] PATH_INFO: `/v3/users/fca80ba4214e483691acda7540a33f02` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:35:52.962626 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201db7e1-2041-4e93-ab3e-699cee3cc743 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fca80ba4214e483691acda7540a33f02)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:35:52.962626 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201db7e1-2041-4e93-ab3e-699cee3cc743 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:35:53.352448 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-201db7e1-2041-4e93-ab3e-699cee3cc743 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fca80ba4214e483691acda7540a33f02'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:35:53.459208 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-201db7e1-2041-4e93-ab3e-699cee3cc743 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fca80ba4214e483691acda7540a33f02'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:35:53.460415 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-201db7e1-2041-4e93-ab3e-699cee3cc743 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fca80ba4214e483691acda7540a33f02'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:35:53.481337 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-201db7e1-2041-4e93-ab3e-699cee3cc743 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fca80ba4214e483691acda7540a33f02'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:35:53.488712 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:35:53.509773 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-201db7e1-2041-4e93-ab3e-699cee3cc743 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fca80ba4214e483691acda7540a33f02'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:35:53.528248 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 870/1722] 10.222.0.30 () {58 vars in 1224 bytes} [Tue Sep 6 04:35:52 2022] DELETE /identity/v3/users/fca80ba4214e483691acda7540a33f02 => generated 0 bytes in 979 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 04:35:53.534968 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c07a84-9c07-404c-b533-ce265733f329 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:35:53.535578 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c07a84-9c07-404c-b533-ce265733f329 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:35:53.535578 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c07a84-9c07-404c-b533-ce265733f329 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:35:53.598498 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-a1c07a84-9c07-404c-b533-ce265733f329 None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:35:53.606460 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-a1c07a84-9c07-404c-b533-ce265733f329 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:35:53.720605 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 853/1723] 10.222.0.30 () {58 vars in 958 bytes} [Tue Sep 6 04:35:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 04:35:53.765266 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201db7e1-2041-4e93-ab3e-699cee3cc743 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:35:53.767001 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-201db7e1-2041-4e93-ab3e-699cee3cc743 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:35:53.866903 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201db7e1-2041-4e93-ab3e-699cee3cc743 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:35:53.873513 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68bc32dd-b813-4a75-838f-c541f6cebc8d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90492440f6a6471e8392361c098428aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b707733ce65c42edbc1fd95ad0a1c06c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:35:53.876125 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bc32dd-b813-4a75-838f-c541f6cebc8d service neutron] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:35:53.876502 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bc32dd-b813-4a75-838f-c541f6cebc8d service neutron] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:35:53.876783 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bc32dd-b813-4a75-838f-c541f6cebc8d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:35:53.893383 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68bc32dd-b813-4a75-838f-c541f6cebc8d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:35:53.896007 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68bc32dd-b813-4a75-838f-c541f6cebc8d service neutron] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:35:53.918537 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 871/1724] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Sep 6 04:35:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 04:35:53.934119 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c07a84-9c07-404c-b533-ce265733f329 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:35:53.935633 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-a1c07a84-9c07-404c-b533-ce265733f329 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:35:53.950028 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c07a84-9c07-404c-b533-ce265733f329 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:35:53.965669 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e1eb88-1922-49af-9f2a-26f5703ac320 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:35:53.966617 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e1eb88-1922-49af-9f2a-26f5703ac320 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:35:53.967113 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e1eb88-1922-49af-9f2a-26f5703ac320 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:35:53.967542 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e1eb88-1922-49af-9f2a-26f5703ac320 admin admin] PATH_INFO: `/v3/users/eaa8037d5b674905b7a31cf11dd5791d` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:35:53.969152 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e1eb88-1922-49af-9f2a-26f5703ac320 admin admin] RBAC: Authorizing `identity:delete_user(user_id=eaa8037d5b674905b7a31cf11dd5791d)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:35:53.979551 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e1eb88-1922-49af-9f2a-26f5703ac320 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:35:54.181261 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-28e1eb88-1922-49af-9f2a-26f5703ac320 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eaa8037d5b674905b7a31cf11dd5791d'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:35:54.300362 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-28e1eb88-1922-49af-9f2a-26f5703ac320 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eaa8037d5b674905b7a31cf11dd5791d'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:35:54.300998 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-28e1eb88-1922-49af-9f2a-26f5703ac320 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eaa8037d5b674905b7a31cf11dd5791d'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:35:54.312119 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-28e1eb88-1922-49af-9f2a-26f5703ac320 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eaa8037d5b674905b7a31cf11dd5791d'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:35:54.343829 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-28e1eb88-1922-49af-9f2a-26f5703ac320 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eaa8037d5b674905b7a31cf11dd5791d'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:35:54.373569 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 854/1725] 10.222.0.30 () {58 vars in 1224 bytes} [Tue Sep 6 04:35:53 2022] DELETE /identity/v3/users/eaa8037d5b674905b7a31cf11dd5791d => generated 0 bytes in 440 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 04:35:54.383865 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd327796-7d7b-4258-b636-77895dd2972f None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:35:54.384437 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd327796-7d7b-4258-b636-77895dd2972f None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:35:54.384898 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd327796-7d7b-4258-b636-77895dd2972f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:35:54.419918 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-dd327796-7d7b-4258-b636-77895dd2972f None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:35:54.424118 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-dd327796-7d7b-4258-b636-77895dd2972f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:35:54.452603 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e1eb88-1922-49af-9f2a-26f5703ac320 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:35:54.455466 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-28e1eb88-1922-49af-9f2a-26f5703ac320 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:35:54.637931 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 872/1726] 10.222.0.30 () {58 vars in 958 bytes} [Tue Sep 6 04:35:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 04:35:54.638433 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e1eb88-1922-49af-9f2a-26f5703ac320 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:35:54.638433 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abf91e3-67d6-4fe0-921c-8f66f73874f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:35:54.638433 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abf91e3-67d6-4fe0-921c-8f66f73874f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:35:54.638433 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abf91e3-67d6-4fe0-921c-8f66f73874f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:35:54.639062 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abf91e3-67d6-4fe0-921c-8f66f73874f4 admin admin] PATH_INFO: `/v3/projects/eee1cc17e9634449b3d2d1b8179aab59` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:35:54.639062 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9abf91e3-67d6-4fe0-921c-8f66f73874f4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=eee1cc17e9634449b3d2d1b8179aab59)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:35:54.639898 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9abf91e3-67d6-4fe0-921c-8f66f73874f4 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:35:54.696999 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd327796-7d7b-4258-b636-77895dd2972f None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:35:54.698962 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-dd327796-7d7b-4258-b636-77895dd2972f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:35:54.910074 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd327796-7d7b-4258-b636-77895dd2972f None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:35:54.942756 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-9abf91e3-67d6-4fe0-921c-8f66f73874f4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eee1cc17e9634449b3d2d1b8179aab59', 'request_id': 'req-9abf91e3-67d6-4fe0-921c-8f66f73874f4'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:35:55.090575 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82936a9-f45d-4029-8d1d-8376d7bfa10f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90492440f6a6471e8392361c098428aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b707733ce65c42edbc1fd95ad0a1c06c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:35:55.090575 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82936a9-f45d-4029-8d1d-8376d7bfa10f service neutron] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:35:55.090575 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82936a9-f45d-4029-8d1d-8376d7bfa10f service neutron] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:35:55.090575 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82936a9-f45d-4029-8d1d-8376d7bfa10f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:35:55.091182 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-9abf91e3-67d6-4fe0-921c-8f66f73874f4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eee1cc17e9634449b3d2d1b8179aab59', 'request_id': 'req-9abf91e3-67d6-4fe0-921c-8f66f73874f4'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:35:55.118162 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-9abf91e3-67d6-4fe0-921c-8f66f73874f4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eee1cc17e9634449b3d2d1b8179aab59', 'request_id': 'req-9abf91e3-67d6-4fe0-921c-8f66f73874f4'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:35:55.161359 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-9abf91e3-67d6-4fe0-921c-8f66f73874f4 admin admin] The token cache is being invalidate because project eee1cc17e9634449b3d2d1b8179aab59 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81697) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 04:35:55.161673 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-9abf91e3-67d6-4fe0-921c-8f66f73874f4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:35:55.163823 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82936a9-f45d-4029-8d1d-8376d7bfa10f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:35:55.165422 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82936a9-f45d-4029-8d1d-8376d7bfa10f service neutron] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:35:55.166979 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-9abf91e3-67d6-4fe0-921c-8f66f73874f4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:35:55.169268 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 855/1727] 10.222.0.30 () {58 vars in 1233 bytes} [Tue Sep 6 04:35:54 2022] DELETE /identity/v3/projects/eee1cc17e9634449b3d2d1b8179aab59 => generated 0 bytes in 717 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 04:35:55.170111 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abf91e3-67d6-4fe0-921c-8f66f73874f4 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:35:55.178754 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-9abf91e3-67d6-4fe0-921c-8f66f73874f4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:35:55.183283 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 873/1728] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Sep 6 04:35:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 493 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 04:35:55.189815 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a76262-385c-488a-8aee-2e8b0102ddc7 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:35:55.189815 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a76262-385c-488a-8aee-2e8b0102ddc7 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:35:55.189815 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a76262-385c-488a-8aee-2e8b0102ddc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:35:55.211835 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abf91e3-67d6-4fe0-921c-8f66f73874f4 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:35:55.233977 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084569b0-0d2a-4d07-98fb-d08b01c06c0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:35:55.233977 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084569b0-0d2a-4d07-98fb-d08b01c06c0d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:35:55.233977 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084569b0-0d2a-4d07-98fb-d08b01c06c0d admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:35:55.233977 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084569b0-0d2a-4d07-98fb-d08b01c06c0d admin admin] PATH_INFO: `/v3/projects/b4f8538b00954b3dacfd9ff9bead0e41` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:35:55.235978 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-084569b0-0d2a-4d07-98fb-d08b01c06c0d admin admin] RBAC: Authorizing `identity:delete_project(project_id=b4f8538b00954b3dacfd9ff9bead0e41)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:35:55.238312 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-d8a76262-385c-488a-8aee-2e8b0102ddc7 None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:35:55.239555 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-d8a76262-385c-488a-8aee-2e8b0102ddc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:35:55.241114 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-084569b0-0d2a-4d07-98fb-d08b01c06c0d admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:35:55.270688 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 874/1729] 10.222.0.30 () {58 vars in 958 bytes} [Tue Sep 6 04:35:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 04:35:55.273401 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aed1af2-0673-4143-8f19-9a43167461d9 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:35:55.274289 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aed1af2-0673-4143-8f19-9a43167461d9 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:35:55.274798 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aed1af2-0673-4143-8f19-9a43167461d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:35:55.308732 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-1aed1af2-0673-4143-8f19-9a43167461d9 None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:35:55.313957 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-1aed1af2-0673-4143-8f19-9a43167461d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:35:55.336030 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 875/1730] 10.222.0.30 () {58 vars in 958 bytes} [Tue Sep 6 04:35:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 04:35:55.337872 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aed1af2-0673-4143-8f19-9a43167461d9 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:35:55.365075 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aed1af2-0673-4143-8f19-9a43167461d9 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:35:55.383152 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b80fafc-9dca-47fd-8abe-90414309f236 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:35:55.383800 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b80fafc-9dca-47fd-8abe-90414309f236 admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:35:55.384081 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b80fafc-9dca-47fd-8abe-90414309f236 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:35:55.384356 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b80fafc-9dca-47fd-8abe-90414309f236 admin admin] PATH_INFO: `/v3/domains` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:35:55.385026 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b80fafc-9dca-47fd-8abe-90414309f236 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:35:55.385525 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b80fafc-9dca-47fd-8abe-90414309f236 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:35:55.387726 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b80fafc-9dca-47fd-8abe-90414309f236 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:35:55.421744 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 876/1731] 10.222.0.30 () {58 vars in 1153 bytes} [Tue Sep 6 04:35:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:35:55.426912 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b80fafc-9dca-47fd-8abe-90414309f236 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:35:55.456013 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b80fafc-9dca-47fd-8abe-90414309f236 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:35:55.459175 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-084569b0-0d2a-4d07-98fb-d08b01c06c0d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b4f8538b00954b3dacfd9ff9bead0e41', 'request_id': 'req-084569b0-0d2a-4d07-98fb-d08b01c06c0d'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:35:55.612651 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3335af99-86cf-4b32-8f19-000e1c42f1c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:35:55.614402 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3335af99-86cf-4b32-8f19-000e1c42f1c9 admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:35:55.614790 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3335af99-86cf-4b32-8f19-000e1c42f1c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:35:55.614977 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3335af99-86cf-4b32-8f19-000e1c42f1c9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:35:55.619394 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3335af99-86cf-4b32-8f19-000e1c42f1c9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:35:55.624852 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3335af99-86cf-4b32-8f19-000e1c42f1c9 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:35:55.705182 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-084569b0-0d2a-4d07-98fb-d08b01c06c0d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b4f8538b00954b3dacfd9ff9bead0e41', 'request_id': 'req-084569b0-0d2a-4d07-98fb-d08b01c06c0d'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:35:55.712675 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-084569b0-0d2a-4d07-98fb-d08b01c06c0d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b4f8538b00954b3dacfd9ff9bead0e41', 'request_id': 'req-084569b0-0d2a-4d07-98fb-d08b01c06c0d'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:35:55.724722 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-084569b0-0d2a-4d07-98fb-d08b01c06c0d admin admin] The token cache is being invalidate because project b4f8538b00954b3dacfd9ff9bead0e41 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81697) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 04:35:55.730997 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-084569b0-0d2a-4d07-98fb-d08b01c06c0d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:35:55.734129 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-084569b0-0d2a-4d07-98fb-d08b01c06c0d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:35:55.736406 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 856/1732] 10.222.0.30 () {58 vars in 1233 bytes} [Tue Sep 6 04:35:55 2022] DELETE /identity/v3/projects/b4f8538b00954b3dacfd9ff9bead0e41 => generated 0 bytes in 566 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 04:35:55.752672 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd44e3ef-e856-44c7-bcca-676b572066ef None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:35:55.754628 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd44e3ef-e856-44c7-bcca-676b572066ef None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:35:55.754998 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd44e3ef-e856-44c7-bcca-676b572066ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:35:55.761932 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 877/1733] 10.222.0.30 () {60 vars in 1153 bytes} [Tue Sep 6 04:35:55 2022] POST /identity/v3/projects => generated 369 bytes in 335 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 04:35:55.763166 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3335af99-86cf-4b32-8f19-000e1c42f1c9 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:35:55.766643 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-3335af99-86cf-4b32-8f19-000e1c42f1c9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:35:55.821510 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-dd44e3ef-e856-44c7-bcca-676b572066ef None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:35:55.822705 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-dd44e3ef-e856-44c7-bcca-676b572066ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:35:55.954372 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3335af99-86cf-4b32-8f19-000e1c42f1c9 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:35:55.975670 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238d46e7-a016-4534-9cf8-754ad255eadb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:35:55.977345 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238d46e7-a016-4534-9cf8-754ad255eadb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:35:55.977938 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238d46e7-a016-4534-9cf8-754ad255eadb admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:35:55.978264 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238d46e7-a016-4534-9cf8-754ad255eadb admin admin] PATH_INFO: `/v3/projects/e419b4601a80491d83a39dbf190f75c1` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:35:55.996305 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238d46e7-a016-4534-9cf8-754ad255eadb admin admin] RBAC: Authorizing `identity:delete_project(project_id=e419b4601a80491d83a39dbf190f75c1)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:35:56.001520 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238d46e7-a016-4534-9cf8-754ad255eadb admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:35:56.014962 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 857/1734] 10.222.0.30 () {58 vars in 958 bytes} [Tue Sep 6 04:35:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 265 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 04:35:56.020054 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd44e3ef-e856-44c7-bcca-676b572066ef None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:35:56.022248 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-dd44e3ef-e856-44c7-bcca-676b572066ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:35:56.056919 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd44e3ef-e856-44c7-bcca-676b572066ef None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:35:56.082228 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeae9202-14c5-4a1b-b822-3cf65d76efc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:35:56.085450 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeae9202-14c5-4a1b-b822-3cf65d76efc8 admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:35:56.085450 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeae9202-14c5-4a1b-b822-3cf65d76efc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:35:56.085450 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeae9202-14c5-4a1b-b822-3cf65d76efc8 admin admin] PATH_INFO: `/v3/users` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:35:56.085450 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeae9202-14c5-4a1b-b822-3cf65d76efc8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:35:56.087022 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeae9202-14c5-4a1b-b822-3cf65d76efc8 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:35:56.185433 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:35:56.190790 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-238d46e7-a016-4534-9cf8-754ad255eadb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e419b4601a80491d83a39dbf190f75c1', 'request_id': 'req-238d46e7-a016-4534-9cf8-754ad255eadb'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:35:56.209166 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-238d46e7-a016-4534-9cf8-754ad255eadb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e419b4601a80491d83a39dbf190f75c1', 'request_id': 'req-238d46e7-a016-4534-9cf8-754ad255eadb'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:35:56.254119 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 858/1735] 10.222.0.30 () {60 vars in 1144 bytes} [Tue Sep 6 04:35:56 2022] POST /identity/v3/users => generated 285 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 04:35:56.255684 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeae9202-14c5-4a1b-b822-3cf65d76efc8 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:35:56.266947 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-eeae9202-14c5-4a1b-b822-3cf65d76efc8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:35:56.290632 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeae9202-14c5-4a1b-b822-3cf65d76efc8 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:35:56.297725 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-238d46e7-a016-4534-9cf8-754ad255eadb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e419b4601a80491d83a39dbf190f75c1', 'request_id': 'req-238d46e7-a016-4534-9cf8-754ad255eadb'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:35:56.327859 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-238d46e7-a016-4534-9cf8-754ad255eadb admin admin] The token cache is being invalidate because project e419b4601a80491d83a39dbf190f75c1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81698) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 04:35:56.328449 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-238d46e7-a016-4534-9cf8-754ad255eadb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:35:56.329163 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-238d46e7-a016-4534-9cf8-754ad255eadb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:35:56.333505 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 878/1736] 10.222.0.30 () {58 vars in 1233 bytes} [Tue Sep 6 04:35:55 2022] DELETE /identity/v3/projects/e419b4601a80491d83a39dbf190f75c1 => generated 0 bytes in 571 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 04:35:56.334923 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edb3b97-3264-4d15-ab66-df5420c832f9 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:35:56.335280 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edb3b97-3264-4d15-ab66-df5420c832f9 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:35:56.336297 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edb3b97-3264-4d15-ab66-df5420c832f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:35:56.398337 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60718150-a51c-4607-a2e7-086c2ea0dd5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:35:56.399393 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60718150-a51c-4607-a2e7-086c2ea0dd5c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:35:56.399741 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60718150-a51c-4607-a2e7-086c2ea0dd5c admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:35:56.400084 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60718150-a51c-4607-a2e7-086c2ea0dd5c admin admin] PATH_INFO: `/v3/projects/6d29ea41356640f9871d15a5dfe8a42d` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:35:56.401609 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60718150-a51c-4607-a2e7-086c2ea0dd5c admin admin] RBAC: Authorizing `identity:delete_project(project_id=6d29ea41356640f9871d15a5dfe8a42d)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:35:56.406290 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60718150-a51c-4607-a2e7-086c2ea0dd5c admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:35:56.415078 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-5edb3b97-3264-4d15-ab66-df5420c832f9 None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:35:56.415078 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-5edb3b97-3264-4d15-ab66-df5420c832f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:35:56.422852 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 879/1737] 10.222.0.30 () {58 vars in 958 bytes} [Tue Sep 6 04:35:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 04:35:56.424358 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5edb3b97-3264-4d15-ab66-df5420c832f9 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:35:56.426645 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-5edb3b97-3264-4d15-ab66-df5420c832f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:35:56.443609 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5edb3b97-3264-4d15-ab66-df5420c832f9 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:35:56.452736 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd24fdff-e6c0-484b-bee6-e9126b05344c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:35:56.453031 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd24fdff-e6c0-484b-bee6-e9126b05344c admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:35:56.453850 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd24fdff-e6c0-484b-bee6-e9126b05344c admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:35:56.453850 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd24fdff-e6c0-484b-bee6-e9126b05344c admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:35:56.454282 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd24fdff-e6c0-484b-bee6-e9126b05344c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:35:56.456307 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd24fdff-e6c0-484b-bee6-e9126b05344c admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:35:56.475312 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 880/1738] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Sep 6 04:35:56 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 04:35:56.476238 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd24fdff-e6c0-484b-bee6-e9126b05344c admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:35:56.480779 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-cd24fdff-e6c0-484b-bee6-e9126b05344c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:35:56.504900 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd24fdff-e6c0-484b-bee6-e9126b05344c admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:35:56.522226 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6deeace9-6c5b-4bff-aa49-b0fd71131bd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:35:56.522662 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6deeace9-6c5b-4bff-aa49-b0fd71131bd0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:35:56.522865 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6deeace9-6c5b-4bff-aa49-b0fd71131bd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:35:56.523119 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6deeace9-6c5b-4bff-aa49-b0fd71131bd0 admin admin] PATH_INFO: `/v3/users/44f40ae47fd2460282a63de44e415cce` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:35:56.524808 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6deeace9-6c5b-4bff-aa49-b0fd71131bd0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=44f40ae47fd2460282a63de44e415cce)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:35:56.528327 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6deeace9-6c5b-4bff-aa49-b0fd71131bd0 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:35:56.540582 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-60718150-a51c-4607-a2e7-086c2ea0dd5c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6d29ea41356640f9871d15a5dfe8a42d', 'request_id': 'req-60718150-a51c-4607-a2e7-086c2ea0dd5c'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:35:56.613392 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-60718150-a51c-4607-a2e7-086c2ea0dd5c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6d29ea41356640f9871d15a5dfe8a42d', 'request_id': 'req-60718150-a51c-4607-a2e7-086c2ea0dd5c'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:35:56.620781 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-60718150-a51c-4607-a2e7-086c2ea0dd5c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6d29ea41356640f9871d15a5dfe8a42d', 'request_id': 'req-60718150-a51c-4607-a2e7-086c2ea0dd5c'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:35:56.631483 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-60718150-a51c-4607-a2e7-086c2ea0dd5c admin admin] The token cache is being invalidate because project 6d29ea41356640f9871d15a5dfe8a42d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81697) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 04:35:56.631775 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-60718150-a51c-4607-a2e7-086c2ea0dd5c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:35:56.632456 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-60718150-a51c-4607-a2e7-086c2ea0dd5c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:35:56.634001 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 859/1739] 10.222.0.30 () {58 vars in 1233 bytes} [Tue Sep 6 04:35:56 2022] DELETE /identity/v3/projects/6d29ea41356640f9871d15a5dfe8a42d => generated 0 bytes in 379 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 04:35:56.637698 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60718150-a51c-4607-a2e7-086c2ea0dd5c admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:35:56.637698 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-60718150-a51c-4607-a2e7-086c2ea0dd5c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:35:56.704159 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:35:56.735882 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60718150-a51c-4607-a2e7-086c2ea0dd5c admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:35:56.740551 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d211ab0c-c58e-4353-8cdf-f0a780da6965 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:35:56.741171 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d211ab0c-c58e-4353-8cdf-f0a780da6965 admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:35:56.741487 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d211ab0c-c58e-4353-8cdf-f0a780da6965 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:35:56.741765 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d211ab0c-c58e-4353-8cdf-f0a780da6965 admin admin] PATH_INFO: `/v3/domains` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:35:56.742415 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d211ab0c-c58e-4353-8cdf-f0a780da6965 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:35:56.742720 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d211ab0c-c58e-4353-8cdf-f0a780da6965 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:35:56.744349 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d211ab0c-c58e-4353-8cdf-f0a780da6965 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:35:56.760809 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 860/1740] 10.222.0.30 () {58 vars in 1153 bytes} [Tue Sep 6 04:35:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:35:56.760809 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d211ab0c-c58e-4353-8cdf-f0a780da6965 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:35:56.760809 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-d211ab0c-c58e-4353-8cdf-f0a780da6965 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:35:56.773210 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d211ab0c-c58e-4353-8cdf-f0a780da6965 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:35:56.778971 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45b4d8a-ee01-46d2-8e67-0eddbf460b96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:35:56.779587 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45b4d8a-ee01-46d2-8e67-0eddbf460b96 admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:35:56.779871 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45b4d8a-ee01-46d2-8e67-0eddbf460b96 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:35:56.780150 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45b4d8a-ee01-46d2-8e67-0eddbf460b96 admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:35:56.780845 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45b4d8a-ee01-46d2-8e67-0eddbf460b96 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:35:56.782475 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45b4d8a-ee01-46d2-8e67-0eddbf460b96 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:35:56.794333 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 861/1741] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Sep 6 04:35:56 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 04:35:56.798207 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45b4d8a-ee01-46d2-8e67-0eddbf460b96 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:35:56.800411 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-f45b4d8a-ee01-46d2-8e67-0eddbf460b96 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:35:56.813286 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45b4d8a-ee01-46d2-8e67-0eddbf460b96 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:35:56.817876 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b688d8-d4a4-4d13-a3f1-ad257760f273 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:35:56.823386 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b688d8-d4a4-4d13-a3f1-ad257760f273 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:35:56.823386 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b688d8-d4a4-4d13-a3f1-ad257760f273 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:35:56.823386 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b688d8-d4a4-4d13-a3f1-ad257760f273 admin admin] PATH_INFO: `/v3/users/863d0ca50601424fac9c22be0caba6ad` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:35:56.823386 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b688d8-d4a4-4d13-a3f1-ad257760f273 admin admin] RBAC: Authorizing `identity:delete_user(user_id=863d0ca50601424fac9c22be0caba6ad)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:35:56.823386 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b688d8-d4a4-4d13-a3f1-ad257760f273 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:35:56.856388 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-6deeace9-6c5b-4bff-aa49-b0fd71131bd0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '44f40ae47fd2460282a63de44e415cce'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:35:56.950587 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-6deeace9-6c5b-4bff-aa49-b0fd71131bd0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '44f40ae47fd2460282a63de44e415cce'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:35:56.951159 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-6deeace9-6c5b-4bff-aa49-b0fd71131bd0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '44f40ae47fd2460282a63de44e415cce'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:35:56.964733 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-6deeace9-6c5b-4bff-aa49-b0fd71131bd0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '44f40ae47fd2460282a63de44e415cce'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:35:56.971625 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-6deeace9-6c5b-4bff-aa49-b0fd71131bd0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '44f40ae47fd2460282a63de44e415cce'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:35:56.977571 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 881/1742] 10.222.0.30 () {58 vars in 1224 bytes} [Tue Sep 6 04:35:56 2022] DELETE /identity/v3/users/44f40ae47fd2460282a63de44e415cce => generated 0 bytes in 502 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 04:35:56.978434 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6deeace9-6c5b-4bff-aa49-b0fd71131bd0 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:35:56.979781 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-6deeace9-6c5b-4bff-aa49-b0fd71131bd0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:35:57.044866 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6deeace9-6c5b-4bff-aa49-b0fd71131bd0 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:35:57.050165 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89119fab-f917-4bf7-81e8-73beba10886a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:35:57.055124 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89119fab-f917-4bf7-81e8-73beba10886a admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:35:57.055124 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89119fab-f917-4bf7-81e8-73beba10886a admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:35:57.055124 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89119fab-f917-4bf7-81e8-73beba10886a admin admin] PATH_INFO: `/v3/projects` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:35:57.055124 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89119fab-f917-4bf7-81e8-73beba10886a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:35:57.055124 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89119fab-f917-4bf7-81e8-73beba10886a admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:35:57.066046 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-68b688d8-d4a4-4d13-a3f1-ad257760f273 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '863d0ca50601424fac9c22be0caba6ad'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:35:57.171837 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-68b688d8-d4a4-4d13-a3f1-ad257760f273 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '863d0ca50601424fac9c22be0caba6ad'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:35:57.172440 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-68b688d8-d4a4-4d13-a3f1-ad257760f273 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '863d0ca50601424fac9c22be0caba6ad'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:35:57.184550 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-68b688d8-d4a4-4d13-a3f1-ad257760f273 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '863d0ca50601424fac9c22be0caba6ad'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:35:57.184723 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 882/1743] 10.222.0.30 () {60 vars in 1153 bytes} [Tue Sep 6 04:35:56 2022] POST /identity/v3/projects => generated 357 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 04:35:57.184723 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89119fab-f917-4bf7-81e8-73beba10886a admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:35:57.192727 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-89119fab-f917-4bf7-81e8-73beba10886a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:35:57.196567 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-68b688d8-d4a4-4d13-a3f1-ad257760f273 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '863d0ca50601424fac9c22be0caba6ad'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:35:57.206731 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 862/1744] 10.222.0.30 () {58 vars in 1224 bytes} [Tue Sep 6 04:35:56 2022] DELETE /identity/v3/users/863d0ca50601424fac9c22be0caba6ad => generated 0 bytes in 409 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 04:35:57.206731 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b688d8-d4a4-4d13-a3f1-ad257760f273 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:35:57.208235 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-68b688d8-d4a4-4d13-a3f1-ad257760f273 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:35:57.295606 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89119fab-f917-4bf7-81e8-73beba10886a admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:35:57.300649 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b688d8-d4a4-4d13-a3f1-ad257760f273 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:35:57.301496 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31119c40-3073-4d2b-a13b-b2bb495a508e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:35:57.302271 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31119c40-3073-4d2b-a13b-b2bb495a508e admin admin] REQUEST_METHOD: `PUT` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:35:57.302539 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31119c40-3073-4d2b-a13b-b2bb495a508e admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:35:57.303392 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31119c40-3073-4d2b-a13b-b2bb495a508e admin admin] PATH_INFO: `/v3/projects/422cae1387de42c9b2379a9ab95e586a/users/79fe4ce702654b9a957544d55e5df57c/roles/29607b2de39f40d68bb6ceef64b8a732` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:35:57.307699 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b3e1ca-bef1-424a-9e9c-f83bef0dd461 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:35:57.308645 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b3e1ca-bef1-424a-9e9c-f83bef0dd461 admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:35:57.309227 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b3e1ca-bef1-424a-9e9c-f83bef0dd461 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:35:57.309349 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b3e1ca-bef1-424a-9e9c-f83bef0dd461 admin admin] PATH_INFO: `/v3/users` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:35:57.309938 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b3e1ca-bef1-424a-9e9c-f83bef0dd461 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:35:57.311887 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b3e1ca-bef1-424a-9e9c-f83bef0dd461 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:35:57.324120 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31119c40-3073-4d2b-a13b-b2bb495a508e admin admin] RBAC: Authorizing `identity:create_grant(project_id=422cae1387de42c9b2379a9ab95e586a, user_id=79fe4ce702654b9a957544d55e5df57c, role_id=29607b2de39f40d68bb6ceef64b8a732)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:35:57.327481 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31119c40-3073-4d2b-a13b-b2bb495a508e admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:35:57.413172 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 883/1745] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Sep 6 04:35:57 2022] PUT /identity/v3/projects/422cae1387de42c9b2379a9ab95e586a/users/79fe4ce702654b9a957544d55e5df57c/roles/29607b2de39f40d68bb6ceef64b8a732 => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 04:35:57.415516 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 863/1746] 10.222.0.30 () {60 vars in 1144 bytes} [Tue Sep 6 04:35:57 2022] POST /identity/v3/users => generated 279 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 04:35:57.418513 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31119c40-3073-4d2b-a13b-b2bb495a508e admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:35:57.424101 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df1c7c8-0c8e-4d2f-a85b-fdef5f98739a None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:35:57.424672 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df1c7c8-0c8e-4d2f-a85b-fdef5f98739a None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:35:57.425062 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df1c7c8-0c8e-4d2f-a85b-fdef5f98739a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:35:57.458555 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-5df1c7c8-0c8e-4d2f-a85b-fdef5f98739a None None] MFA Rules not processed for user `79fe4ce702654b9a957544d55e5df57c`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:35:57.459891 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-5df1c7c8-0c8e-4d2f-a85b-fdef5f98739a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:35:57.507160 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31119c40-3073-4d2b-a13b-b2bb495a508e admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:35:57.537144 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609790a2-914f-4a00-ba0f-80a37b0f4d85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:35:57.541073 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609790a2-914f-4a00-ba0f-80a37b0f4d85 admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:35:57.542789 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609790a2-914f-4a00-ba0f-80a37b0f4d85 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:35:57.543838 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 864/1747] 10.222.0.30 () {58 vars in 958 bytes} [Tue Sep 6 04:35:57 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 04:35:57.544493 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609790a2-914f-4a00-ba0f-80a37b0f4d85 admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:35:57.545362 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609790a2-914f-4a00-ba0f-80a37b0f4d85 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:35:57.547195 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609790a2-914f-4a00-ba0f-80a37b0f4d85 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:35:57.548530 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df1c7c8-0c8e-4d2f-a85b-fdef5f98739a None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:35:57.550242 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-5df1c7c8-0c8e-4d2f-a85b-fdef5f98739a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:35:57.562017 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 884/1748] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Sep 6 04:35:57 2022] GET /identity/v3/roles => generated 1420 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 06 04:35:57.562845 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609790a2-914f-4a00-ba0f-80a37b0f4d85 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:35:57.573591 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-609790a2-914f-4a00-ba0f-80a37b0f4d85 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:35:57.574947 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df1c7c8-0c8e-4d2f-a85b-fdef5f98739a None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:35:57.580495 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6d77bb-5853-4895-ab6e-1ec0c8ab9b27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:35:57.581145 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6d77bb-5853-4895-ab6e-1ec0c8ab9b27 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:35:57.581461 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6d77bb-5853-4895-ab6e-1ec0c8ab9b27 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:35:57.581731 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6d77bb-5853-4895-ab6e-1ec0c8ab9b27 admin admin] PATH_INFO: `/v3/projects/95bedd195b214bfab139a44b3e770e0c` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:35:57.582904 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6d77bb-5853-4895-ab6e-1ec0c8ab9b27 admin admin] RBAC: Authorizing `identity:delete_project(project_id=95bedd195b214bfab139a44b3e770e0c)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:35:57.584846 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6d77bb-5853-4895-ab6e-1ec0c8ab9b27 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:35:57.586746 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609790a2-914f-4a00-ba0f-80a37b0f4d85 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:35:57.593821 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e4eb46-bd55-45bb-89e6-096df11d86ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:35:57.595781 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e4eb46-bd55-45bb-89e6-096df11d86ed admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:35:57.596644 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e4eb46-bd55-45bb-89e6-096df11d86ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:35:57.597926 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e4eb46-bd55-45bb-89e6-096df11d86ed admin admin] PATH_INFO: `/v3/projects` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:35:57.604947 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e4eb46-bd55-45bb-89e6-096df11d86ed admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:35:57.604947 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e4eb46-bd55-45bb-89e6-096df11d86ed admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:35:57.725500 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-2f6d77bb-5853-4895-ab6e-1ec0c8ab9b27 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '95bedd195b214bfab139a44b3e770e0c', 'request_id': 'req-2f6d77bb-5853-4895-ab6e-1ec0c8ab9b27'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:35:57.730495 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 885/1749] 10.222.0.30 () {60 vars in 1153 bytes} [Tue Sep 6 04:35:57 2022] POST /identity/v3/projects => generated 367 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 04:35:57.731244 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e4eb46-bd55-45bb-89e6-096df11d86ed admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:35:57.804767 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e4eb46-bd55-45bb-89e6-096df11d86ed admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:35:57.809846 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4066925e-3c54-4b21-a9b6-6f1aafb73f68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:35:57.810428 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4066925e-3c54-4b21-a9b6-6f1aafb73f68 admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:35:57.810711 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4066925e-3c54-4b21-a9b6-6f1aafb73f68 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:35:57.810985 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4066925e-3c54-4b21-a9b6-6f1aafb73f68 admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:35:57.811686 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4066925e-3c54-4b21-a9b6-6f1aafb73f68 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:35:57.813355 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4066925e-3c54-4b21-a9b6-6f1aafb73f68 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:35:57.825694 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 886/1750] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Sep 6 04:35:57 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 04:35:57.826611 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4066925e-3c54-4b21-a9b6-6f1aafb73f68 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:35:57.829332 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-2f6d77bb-5853-4895-ab6e-1ec0c8ab9b27 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '95bedd195b214bfab139a44b3e770e0c', 'request_id': 'req-2f6d77bb-5853-4895-ab6e-1ec0c8ab9b27'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:35:57.836318 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-2f6d77bb-5853-4895-ab6e-1ec0c8ab9b27 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '95bedd195b214bfab139a44b3e770e0c', 'request_id': 'req-2f6d77bb-5853-4895-ab6e-1ec0c8ab9b27'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:35:57.843932 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4066925e-3c54-4b21-a9b6-6f1aafb73f68 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:35:57.847385 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-2f6d77bb-5853-4895-ab6e-1ec0c8ab9b27 admin admin] The token cache is being invalidate because project 95bedd195b214bfab139a44b3e770e0c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81697) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 04:35:57.847780 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-2f6d77bb-5853-4895-ab6e-1ec0c8ab9b27 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:35:57.848528 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-2f6d77bb-5853-4895-ab6e-1ec0c8ab9b27 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:35:57.848926 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729a9eb7-50cf-47fd-8ef0-43de67c102d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:35:57.849535 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729a9eb7-50cf-47fd-8ef0-43de67c102d3 admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:35:57.850423 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729a9eb7-50cf-47fd-8ef0-43de67c102d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:35:57.850423 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729a9eb7-50cf-47fd-8ef0-43de67c102d3 admin admin] PATH_INFO: `/v3/users` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:35:57.850901 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729a9eb7-50cf-47fd-8ef0-43de67c102d3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:35:57.852840 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729a9eb7-50cf-47fd-8ef0-43de67c102d3 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:35:57.853438 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 865/1751] 10.222.0.30 () {58 vars in 1233 bytes} [Tue Sep 6 04:35:57 2022] DELETE /identity/v3/projects/95bedd195b214bfab139a44b3e770e0c => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 04:35:57.854488 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6d77bb-5853-4895-ab6e-1ec0c8ab9b27 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:35:57.856390 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-2f6d77bb-5853-4895-ab6e-1ec0c8ab9b27 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:35:57.924531 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6d77bb-5853-4895-ab6e-1ec0c8ab9b27 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:35:57.929553 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4360bbb3-91c1-47b1-ba09-91f923de440d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:35:57.930274 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4360bbb3-91c1-47b1-ba09-91f923de440d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:35:57.930621 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4360bbb3-91c1-47b1-ba09-91f923de440d admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:35:57.930962 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4360bbb3-91c1-47b1-ba09-91f923de440d admin admin] PATH_INFO: `/v3/projects/1e6259212e5648f4b668a7d84767bacd` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:35:57.932262 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4360bbb3-91c1-47b1-ba09-91f923de440d admin admin] RBAC: Authorizing `identity:delete_project(project_id=1e6259212e5648f4b668a7d84767bacd)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:35:57.934407 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4360bbb3-91c1-47b1-ba09-91f923de440d admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:35:57.974017 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 887/1752] 10.222.0.30 () {60 vars in 1144 bytes} [Tue Sep 6 04:35:57 2022] POST /identity/v3/users => generated 290 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 04:35:57.974512 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729a9eb7-50cf-47fd-8ef0-43de67c102d3 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:35:57.976308 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-729a9eb7-50cf-47fd-8ef0-43de67c102d3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:35:57.990384 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729a9eb7-50cf-47fd-8ef0-43de67c102d3 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:35:57.995640 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ed4f75-06c1-4e72-a99d-0d4f7dcd9832 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:35:57.996307 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ed4f75-06c1-4e72-a99d-0d4f7dcd9832 admin admin] REQUEST_METHOD: `PUT` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:35:57.996590 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ed4f75-06c1-4e72-a99d-0d4f7dcd9832 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:35:57.996866 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ed4f75-06c1-4e72-a99d-0d4f7dcd9832 admin admin] PATH_INFO: `/v3/projects/706079ad6e194baf9bcdc5daee82a949/users/c87bf43bae59424691dab57abc4a685b/roles/29607b2de39f40d68bb6ceef64b8a732` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:35:58.012455 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ed4f75-06c1-4e72-a99d-0d4f7dcd9832 admin admin] RBAC: Authorizing `identity:create_grant(project_id=706079ad6e194baf9bcdc5daee82a949, user_id=c87bf43bae59424691dab57abc4a685b, role_id=29607b2de39f40d68bb6ceef64b8a732)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:35:58.015178 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ed4f75-06c1-4e72-a99d-0d4f7dcd9832 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:35:58.091369 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 888/1753] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Sep 6 04:35:57 2022] PUT /identity/v3/projects/706079ad6e194baf9bcdc5daee82a949/users/c87bf43bae59424691dab57abc4a685b/roles/29607b2de39f40d68bb6ceef64b8a732 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 04:35:58.092672 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ed4f75-06c1-4e72-a99d-0d4f7dcd9832 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:35:58.094594 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-22ed4f75-06c1-4e72-a99d-0d4f7dcd9832 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:35:58.108519 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-4360bbb3-91c1-47b1-ba09-91f923de440d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1e6259212e5648f4b668a7d84767bacd', 'request_id': 'req-4360bbb3-91c1-47b1-ba09-91f923de440d'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:35:58.161171 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ed4f75-06c1-4e72-a99d-0d4f7dcd9832 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:35:58.201701 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-4360bbb3-91c1-47b1-ba09-91f923de440d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1e6259212e5648f4b668a7d84767bacd', 'request_id': 'req-4360bbb3-91c1-47b1-ba09-91f923de440d'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:35:58.208372 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-4360bbb3-91c1-47b1-ba09-91f923de440d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1e6259212e5648f4b668a7d84767bacd', 'request_id': 'req-4360bbb3-91c1-47b1-ba09-91f923de440d'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:35:58.216603 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da54e03-be4b-409c-8d31-553fd58a2540 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:35:58.217426 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da54e03-be4b-409c-8d31-553fd58a2540 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:35:58.217954 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da54e03-be4b-409c-8d31-553fd58a2540 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:35:58.218341 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da54e03-be4b-409c-8d31-553fd58a2540 admin admin] PATH_INFO: `/v3/users/bdef1c0f68b34452877b2123825d6dd2` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:35:58.219578 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da54e03-be4b-409c-8d31-553fd58a2540 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bdef1c0f68b34452877b2123825d6dd2)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:35:58.220025 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-4360bbb3-91c1-47b1-ba09-91f923de440d admin admin] The token cache is being invalidate because project 1e6259212e5648f4b668a7d84767bacd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81697) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 04:35:58.220325 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-4360bbb3-91c1-47b1-ba09-91f923de440d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:35:58.221000 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-4360bbb3-91c1-47b1-ba09-91f923de440d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:35:58.221568 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da54e03-be4b-409c-8d31-553fd58a2540 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:35:58.222805 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 866/1754] 10.222.0.30 () {58 vars in 1233 bytes} [Tue Sep 6 04:35:57 2022] DELETE /identity/v3/projects/1e6259212e5648f4b668a7d84767bacd => generated 0 bytes in 369 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 04:35:58.223340 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4360bbb3-91c1-47b1-ba09-91f923de440d admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:35:58.231428 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-4360bbb3-91c1-47b1-ba09-91f923de440d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:35:58.240604 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4360bbb3-91c1-47b1-ba09-91f923de440d admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:35:58.246033 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300de64e-ba91-4392-9868-898f637dc6af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:35:58.246890 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300de64e-ba91-4392-9868-898f637dc6af admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:35:58.247014 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300de64e-ba91-4392-9868-898f637dc6af admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:35:58.247259 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300de64e-ba91-4392-9868-898f637dc6af admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:35:58.247972 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300de64e-ba91-4392-9868-898f637dc6af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:35:58.249663 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300de64e-ba91-4392-9868-898f637dc6af admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:35:58.261997 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 867/1755] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Sep 6 04:35:58 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 04:35:58.263253 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cda6d8f-268e-46ee-8df8-45e8b2068842 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:35:58.263534 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cda6d8f-268e-46ee-8df8-45e8b2068842 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:35:58.263824 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cda6d8f-268e-46ee-8df8-45e8b2068842 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:35:58.293462 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-5cda6d8f-268e-46ee-8df8-45e8b2068842 None None] MFA Rules not processed for user `c87bf43bae59424691dab57abc4a685b`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:35:58.294712 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-5cda6d8f-268e-46ee-8df8-45e8b2068842 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:35:58.363114 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 868/1756] 10.222.0.30 () {58 vars in 958 bytes} [Tue Sep 6 04:35:58 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 04:35:58.364398 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1273e9-a457-4196-b469-5e2926f9aa51 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:35:58.364690 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1273e9-a457-4196-b469-5e2926f9aa51 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:35:58.364969 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1273e9-a457-4196-b469-5e2926f9aa51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:35:58.382091 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-6da54e03-be4b-409c-8d31-553fd58a2540 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bdef1c0f68b34452877b2123825d6dd2'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:35:58.386747 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-da1273e9-a457-4196-b469-5e2926f9aa51 None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:35:58.391823 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-da1273e9-a457-4196-b469-5e2926f9aa51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:35:58.396163 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 869/1757] 10.222.0.30 () {58 vars in 958 bytes} [Tue Sep 6 04:35:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 04:35:58.396916 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1273e9-a457-4196-b469-5e2926f9aa51 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:35:58.407091 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1273e9-a457-4196-b469-5e2926f9aa51 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:35:58.411921 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b08c85-e6de-457d-a489-b659d465c54e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:35:58.412579 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b08c85-e6de-457d-a489-b659d465c54e admin admin] REQUEST_METHOD: `PUT` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:35:58.412856 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b08c85-e6de-457d-a489-b659d465c54e admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:35:58.413126 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b08c85-e6de-457d-a489-b659d465c54e admin admin] PATH_INFO: `/v3/projects/20d9bb2cd54c4208a94d17548633f468/users/00509315822d4e4380ea31abc5403343/roles/ea4c814713b149108e488de855b025e4` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:35:58.428868 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b08c85-e6de-457d-a489-b659d465c54e admin admin] RBAC: Authorizing `identity:create_grant(project_id=20d9bb2cd54c4208a94d17548633f468, user_id=00509315822d4e4380ea31abc5403343, role_id=ea4c814713b149108e488de855b025e4)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:35:58.431566 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b08c85-e6de-457d-a489-b659d465c54e admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:35:58.470825 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-6da54e03-be4b-409c-8d31-553fd58a2540 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bdef1c0f68b34452877b2123825d6dd2'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:35:58.471469 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-6da54e03-be4b-409c-8d31-553fd58a2540 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bdef1c0f68b34452877b2123825d6dd2'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:35:58.484711 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-6da54e03-be4b-409c-8d31-553fd58a2540 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bdef1c0f68b34452877b2123825d6dd2'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:35:58.491440 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-6da54e03-be4b-409c-8d31-553fd58a2540 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bdef1c0f68b34452877b2123825d6dd2'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:35:58.498382 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 889/1758] 10.222.0.30 () {58 vars in 1224 bytes} [Tue Sep 6 04:35:58 2022] DELETE /identity/v3/users/bdef1c0f68b34452877b2123825d6dd2 => generated 0 bytes in 406 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 04:35:58.499355 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da54e03-be4b-409c-8d31-553fd58a2540 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:35:58.500759 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-6da54e03-be4b-409c-8d31-553fd58a2540 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:35:58.534567 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 870/1759] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Sep 6 04:35:58 2022] PUT /identity/v3/projects/20d9bb2cd54c4208a94d17548633f468/users/00509315822d4e4380ea31abc5403343/roles/ea4c814713b149108e488de855b025e4 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 04:35:58.535832 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff785ee-ee08-4ca7-8937-e54fba38015c None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:35:58.536125 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff785ee-ee08-4ca7-8937-e54fba38015c None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:35:58.544719 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff785ee-ee08-4ca7-8937-e54fba38015c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:35:58.559345 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-5ff785ee-ee08-4ca7-8937-e54fba38015c None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:35:58.560972 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-5ff785ee-ee08-4ca7-8937-e54fba38015c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:35:58.570140 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da54e03-be4b-409c-8d31-553fd58a2540 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:35:58.585844 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:35:58.630028 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81119b8f-bd30-45e2-8ed7-a0aa30a58348 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:35:58.630681 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81119b8f-bd30-45e2-8ed7-a0aa30a58348 admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:35:58.631651 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81119b8f-bd30-45e2-8ed7-a0aa30a58348 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:35:58.631943 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81119b8f-bd30-45e2-8ed7-a0aa30a58348 admin admin] PATH_INFO: `/v3/projects` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:35:58.632589 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81119b8f-bd30-45e2-8ed7-a0aa30a58348 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:35:58.634588 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81119b8f-bd30-45e2-8ed7-a0aa30a58348 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:35:58.638078 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 871/1760] 10.222.0.30 () {58 vars in 958 bytes} [Tue Sep 6 04:35:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 04:35:58.639158 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff785ee-ee08-4ca7-8937-e54fba38015c None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:35:58.641005 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-5ff785ee-ee08-4ca7-8937-e54fba38015c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:35:58.656344 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff785ee-ee08-4ca7-8937-e54fba38015c None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:35:58.661674 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c414a49d-4d07-40d3-b4ef-2c80f166a65f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:35:58.662289 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c414a49d-4d07-40d3-b4ef-2c80f166a65f admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:35:58.662578 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c414a49d-4d07-40d3-b4ef-2c80f166a65f admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:35:58.662836 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c414a49d-4d07-40d3-b4ef-2c80f166a65f admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:35:58.663637 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c414a49d-4d07-40d3-b4ef-2c80f166a65f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:35:58.665230 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c414a49d-4d07-40d3-b4ef-2c80f166a65f admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:35:58.675598 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 872/1761] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Sep 6 04:35:58 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 04:35:58.676436 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c414a49d-4d07-40d3-b4ef-2c80f166a65f admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:35:58.688377 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c414a49d-4d07-40d3-b4ef-2c80f166a65f admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:35:58.694049 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23822e3-c88e-460e-b1a8-c63222d16830 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:35:58.694049 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23822e3-c88e-460e-b1a8-c63222d16830 admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:35:58.694257 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23822e3-c88e-460e-b1a8-c63222d16830 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:35:58.694500 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23822e3-c88e-460e-b1a8-c63222d16830 admin admin] PATH_INFO: `/v3/domains` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:35:58.695215 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f23822e3-c88e-460e-b1a8-c63222d16830 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:35:58.695515 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f23822e3-c88e-460e-b1a8-c63222d16830 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:35:58.697167 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f23822e3-c88e-460e-b1a8-c63222d16830 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:35:58.710358 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 873/1762] 10.222.0.30 () {58 vars in 1153 bytes} [Tue Sep 6 04:35:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:35:58.711167 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23822e3-c88e-460e-b1a8-c63222d16830 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:35:58.721334 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23822e3-c88e-460e-b1a8-c63222d16830 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:35:58.726982 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1acbea0-a997-43a6-b922-b8fd2fd2a3f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:35:58.727388 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1acbea0-a997-43a6-b922-b8fd2fd2a3f3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:35:58.727672 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1acbea0-a997-43a6-b922-b8fd2fd2a3f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:35:58.727946 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1acbea0-a997-43a6-b922-b8fd2fd2a3f3 admin admin] PATH_INFO: `/v3/projects/20d9bb2cd54c4208a94d17548633f468/users/00509315822d4e4380ea31abc5403343/roles/ea4c814713b149108e488de855b025e4` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:35:58.730086 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1acbea0-a997-43a6-b922-b8fd2fd2a3f3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=20d9bb2cd54c4208a94d17548633f468, user_id=00509315822d4e4380ea31abc5403343, role_id=ea4c814713b149108e488de855b025e4)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:35:58.732817 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1acbea0-a997-43a6-b922-b8fd2fd2a3f3 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:35:58.754863 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 874/1763] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Sep 6 04:35:58 2022] PUT /identity/v3/projects/20d9bb2cd54c4208a94d17548633f468/users/00509315822d4e4380ea31abc5403343/roles/ea4c814713b149108e488de855b025e4 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 04:35:58.754863 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1acbea0-a997-43a6-b922-b8fd2fd2a3f3 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:35:58.755529 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 890/1764] 10.222.0.30 () {60 vars in 1153 bytes} [Tue Sep 6 04:35:58 2022] POST /identity/v3/projects => generated 355 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 04:35:58.759416 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318d7e4d-2cd6-4c91-aab7-90619c75dcc9 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:35:58.759775 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318d7e4d-2cd6-4c91-aab7-90619c75dcc9 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:35:58.760061 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318d7e4d-2cd6-4c91-aab7-90619c75dcc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:35:58.790780 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-318d7e4d-2cd6-4c91-aab7-90619c75dcc9 None None] MFA Rules not processed for user `00509315822d4e4380ea31abc5403343`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:35:58.792035 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-318d7e4d-2cd6-4c91-aab7-90619c75dcc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:35:58.829465 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1acbea0-a997-43a6-b922-b8fd2fd2a3f3 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:35:58.890946 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 891/1765] 10.222.0.30 () {58 vars in 958 bytes} [Tue Sep 6 04:35:58 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 04:35:58.892961 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318d7e4d-2cd6-4c91-aab7-90619c75dcc9 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:35:58.905143 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:35:58.908288 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ee945f-3eac-423f-8815-046e7aef17b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:35:58.911122 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ee945f-3eac-423f-8815-046e7aef17b6 admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:35:58.911122 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ee945f-3eac-423f-8815-046e7aef17b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:35:58.911122 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ee945f-3eac-423f-8815-046e7aef17b6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:35:58.912194 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ee945f-3eac-423f-8815-046e7aef17b6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:35:58.914535 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ee945f-3eac-423f-8815-046e7aef17b6 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:35:58.998006 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318d7e4d-2cd6-4c91-aab7-90619c75dcc9 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:35:59.003250 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ee9815-43e7-4a6c-861e-ffed43e596b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:35:59.003875 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ee9815-43e7-4a6c-861e-ffed43e596b1 admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:35:59.004163 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ee9815-43e7-4a6c-861e-ffed43e596b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:35:59.004440 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ee9815-43e7-4a6c-861e-ffed43e596b1 admin admin] PATH_INFO: `/v3/users` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:35:59.005133 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ee9815-43e7-4a6c-861e-ffed43e596b1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:35:59.007620 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ee9815-43e7-4a6c-861e-ffed43e596b1 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:35:59.029074 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 875/1766] 10.222.0.30 () {60 vars in 1153 bytes} [Tue Sep 6 04:35:58 2022] POST /identity/v3/projects => generated 365 bytes in 284 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 04:35:59.030121 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ee945f-3eac-423f-8815-046e7aef17b6 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:35:59.031799 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-66ee945f-3eac-423f-8815-046e7aef17b6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:35:59.097799 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ee945f-3eac-423f-8815-046e7aef17b6 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:35:59.101862 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34968a8-ffb2-4dd9-876a-0bbdc6507e7a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd09b5ab675634a2eb4b03755743f7d0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b707733ce65c42edbc1fd95ad0a1c06c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:35:59.102479 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34968a8-ffb2-4dd9-876a-0bbdc6507e7a service cinder] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:35:59.102759 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34968a8-ffb2-4dd9-876a-0bbdc6507e7a service cinder] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:35:59.103031 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34968a8-ffb2-4dd9-876a-0bbdc6507e7a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:35:59.115349 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 892/1767] 10.222.0.30 () {60 vars in 1144 bytes} [Tue Sep 6 04:35:58 2022] POST /identity/v3/users => generated 284 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 04:35:59.116514 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ee9815-43e7-4a6c-861e-ffed43e596b1 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:35:59.132217 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ee9815-43e7-4a6c-861e-ffed43e596b1 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:35:59.202121 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34968a8-ffb2-4dd9-876a-0bbdc6507e7a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:35:59.203897 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34968a8-ffb2-4dd9-876a-0bbdc6507e7a service cinder] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:35:59.217131 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 876/1768] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Sep 6 04:35:59 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 187 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 04:35:59.218216 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34968a8-ffb2-4dd9-876a-0bbdc6507e7a service cinder] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:35:59.235915 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d592e8a0-a298-42ed-81cc-5d653c98bd99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:35:59.236854 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d592e8a0-a298-42ed-81cc-5d653c98bd99 admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:35:59.237377 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d592e8a0-a298-42ed-81cc-5d653c98bd99 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:35:59.237872 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d592e8a0-a298-42ed-81cc-5d653c98bd99 admin admin] PATH_INFO: `/v3/users` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:35:59.238740 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d592e8a0-a298-42ed-81cc-5d653c98bd99 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:35:59.245624 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d592e8a0-a298-42ed-81cc-5d653c98bd99 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:35:59.295113 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34968a8-ffb2-4dd9-876a-0bbdc6507e7a service cinder] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:35:59.299669 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac27bb8b-89ed-4351-832f-23d675a6decb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:35:59.300283 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac27bb8b-89ed-4351-832f-23d675a6decb admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:35:59.300561 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac27bb8b-89ed-4351-832f-23d675a6decb admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:35:59.300833 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac27bb8b-89ed-4351-832f-23d675a6decb admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:35:59.301509 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac27bb8b-89ed-4351-832f-23d675a6decb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:35:59.303232 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac27bb8b-89ed-4351-832f-23d675a6decb admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:35:59.314376 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 877/1769] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Sep 6 04:35:59 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 04:35:59.315330 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac27bb8b-89ed-4351-832f-23d675a6decb admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:35:59.317496 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-ac27bb8b-89ed-4351-832f-23d675a6decb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:35:59.330895 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac27bb8b-89ed-4351-832f-23d675a6decb admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:35:59.335697 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0481b38-426a-4c88-9ce6-b1a5ede835c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:35:59.336386 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0481b38-426a-4c88-9ce6-b1a5ede835c3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:35:59.336664 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0481b38-426a-4c88-9ce6-b1a5ede835c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:35:59.336939 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0481b38-426a-4c88-9ce6-b1a5ede835c3 admin admin] PATH_INFO: `/v3/projects/755030d1ad9d4945b341d08a3af30d1a` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:35:59.338129 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0481b38-426a-4c88-9ce6-b1a5ede835c3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=755030d1ad9d4945b341d08a3af30d1a)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:35:59.341033 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0481b38-426a-4c88-9ce6-b1a5ede835c3 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:35:59.394804 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 893/1770] 10.222.0.30 () {60 vars in 1144 bytes} [Tue Sep 6 04:35:59 2022] POST /identity/v3/users => generated 283 bytes in 276 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 04:35:59.394804 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d592e8a0-a298-42ed-81cc-5d653c98bd99 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:35:59.412757 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d592e8a0-a298-42ed-81cc-5d653c98bd99 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:35:59.418249 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01c6176-9aea-4dad-bc41-a1937d35b3e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:35:59.418920 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01c6176-9aea-4dad-bc41-a1937d35b3e4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:35:59.425740 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01c6176-9aea-4dad-bc41-a1937d35b3e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:35:59.425740 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01c6176-9aea-4dad-bc41-a1937d35b3e4 admin admin] PATH_INFO: `/v3/projects/b9576495e4e447d7b953b03118af9614/users/6445823c10e342b0bea67f61383ab645/roles/ea4c814713b149108e488de855b025e4` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:35:59.435819 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01c6176-9aea-4dad-bc41-a1937d35b3e4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b9576495e4e447d7b953b03118af9614, user_id=6445823c10e342b0bea67f61383ab645, role_id=ea4c814713b149108e488de855b025e4)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:35:59.438632 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01c6176-9aea-4dad-bc41-a1937d35b3e4 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:35:59.532314 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 894/1771] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Sep 6 04:35:59 2022] PUT /identity/v3/projects/b9576495e4e447d7b953b03118af9614/users/6445823c10e342b0bea67f61383ab645/roles/ea4c814713b149108e488de855b025e4 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 04:35:59.533265 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01c6176-9aea-4dad-bc41-a1937d35b3e4 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:35:59.540378 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-b0481b38-426a-4c88-9ce6-b1a5ede835c3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '755030d1ad9d4945b341d08a3af30d1a', 'request_id': 'req-b0481b38-426a-4c88-9ce6-b1a5ede835c3'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:35:59.594414 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01c6176-9aea-4dad-bc41-a1937d35b3e4 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:35:59.635180 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-b0481b38-426a-4c88-9ce6-b1a5ede835c3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '755030d1ad9d4945b341d08a3af30d1a', 'request_id': 'req-b0481b38-426a-4c88-9ce6-b1a5ede835c3'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:35:59.644662 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-b0481b38-426a-4c88-9ce6-b1a5ede835c3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '755030d1ad9d4945b341d08a3af30d1a', 'request_id': 'req-b0481b38-426a-4c88-9ce6-b1a5ede835c3'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:35:59.651970 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101803f5-1174-46d9-b8ab-d32fddf7481e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:35:59.652676 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101803f5-1174-46d9-b8ab-d32fddf7481e admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:35:59.652958 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101803f5-1174-46d9-b8ab-d32fddf7481e admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:35:59.653230 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101803f5-1174-46d9-b8ab-d32fddf7481e admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:35:59.653909 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101803f5-1174-46d9-b8ab-d32fddf7481e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:35:59.655506 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101803f5-1174-46d9-b8ab-d32fddf7481e admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:35:59.657153 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-b0481b38-426a-4c88-9ce6-b1a5ede835c3 admin admin] The token cache is being invalidate because project 755030d1ad9d4945b341d08a3af30d1a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81697) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 04:35:59.657484 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-b0481b38-426a-4c88-9ce6-b1a5ede835c3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:35:59.658155 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-b0481b38-426a-4c88-9ce6-b1a5ede835c3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:35:59.659773 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 878/1772] 10.222.0.30 () {58 vars in 1233 bytes} [Tue Sep 6 04:35:59 2022] DELETE /identity/v3/projects/755030d1ad9d4945b341d08a3af30d1a => generated 0 bytes in 345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 04:35:59.660893 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0481b38-426a-4c88-9ce6-b1a5ede835c3 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:35:59.665924 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-b0481b38-426a-4c88-9ce6-b1a5ede835c3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:35:59.670011 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 895/1773] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Sep 6 04:35:59 2022] GET /identity/v3/roles => generated 1420 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 04:35:59.670011 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101803f5-1174-46d9-b8ab-d32fddf7481e admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:35:59.671703 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-101803f5-1174-46d9-b8ab-d32fddf7481e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:35:59.683691 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101803f5-1174-46d9-b8ab-d32fddf7481e admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:35:59.689065 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a2628b-acf9-4c67-b7e5-73d44e27a2d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:35:59.689699 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a2628b-acf9-4c67-b7e5-73d44e27a2d3 admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:35:59.689980 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a2628b-acf9-4c67-b7e5-73d44e27a2d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:35:59.690251 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a2628b-acf9-4c67-b7e5-73d44e27a2d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:35:59.690889 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a2628b-acf9-4c67-b7e5-73d44e27a2d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:35:59.692458 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a2628b-acf9-4c67-b7e5-73d44e27a2d3 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:35:59.704247 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 896/1774] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Sep 6 04:35:59 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 04:35:59.705492 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14df4a3-76d7-4645-87d4-84ff05d92e39 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:35:59.705775 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14df4a3-76d7-4645-87d4-84ff05d92e39 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:35:59.706047 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14df4a3-76d7-4645-87d4-84ff05d92e39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:35:59.726265 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0481b38-426a-4c88-9ce6-b1a5ede835c3 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:35:59.726435 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-d14df4a3-76d7-4645-87d4-84ff05d92e39 None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:35:59.727585 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-d14df4a3-76d7-4645-87d4-84ff05d92e39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:35:59.730206 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d4e7b0-b397-43d3-8355-47fa30856d87 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd09b5ab675634a2eb4b03755743f7d0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b707733ce65c42edbc1fd95ad0a1c06c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:35:59.730826 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d4e7b0-b397-43d3-8355-47fa30856d87 service cinder] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:35:59.731189 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d4e7b0-b397-43d3-8355-47fa30856d87 service cinder] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:35:59.731389 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d4e7b0-b397-43d3-8355-47fa30856d87 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:35:59.733122 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-e0d4e7b0-b397-43d3-8355-47fa30856d87 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:35:59.735901 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 897/1775] 10.222.0.30 () {58 vars in 958 bytes} [Tue Sep 6 04:35:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 04:35:59.736679 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14df4a3-76d7-4645-87d4-84ff05d92e39 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:35:59.737913 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-d14df4a3-76d7-4645-87d4-84ff05d92e39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:35:59.749573 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14df4a3-76d7-4645-87d4-84ff05d92e39 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:35:59.754215 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656fe77d-6291-412f-bf69-c3d2e1120c70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:35:59.754816 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656fe77d-6291-412f-bf69-c3d2e1120c70 admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:35:59.755096 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656fe77d-6291-412f-bf69-c3d2e1120c70 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:35:59.755370 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656fe77d-6291-412f-bf69-c3d2e1120c70 admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:35:59.756010 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656fe77d-6291-412f-bf69-c3d2e1120c70 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:35:59.757644 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656fe77d-6291-412f-bf69-c3d2e1120c70 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:35:59.767872 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 898/1776] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Sep 6 04:35:59 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 04:35:59.768646 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656fe77d-6291-412f-bf69-c3d2e1120c70 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:35:59.778970 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656fe77d-6291-412f-bf69-c3d2e1120c70 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:35:59.783817 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b739cf0-857e-4a86-957b-3b11683a131c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:35:59.784420 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b739cf0-857e-4a86-957b-3b11683a131c admin admin] REQUEST_METHOD: `PUT` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:35:59.784681 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b739cf0-857e-4a86-957b-3b11683a131c admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:35:59.784964 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b739cf0-857e-4a86-957b-3b11683a131c admin admin] PATH_INFO: `/v3/projects/b9576495e4e447d7b953b03118af9614/users/6445823c10e342b0bea67f61383ab645/roles/ea4c814713b149108e488de855b025e4` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:35:59.786943 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b739cf0-857e-4a86-957b-3b11683a131c admin admin] RBAC: Authorizing `identity:create_grant(project_id=b9576495e4e447d7b953b03118af9614, user_id=6445823c10e342b0bea67f61383ab645, role_id=ea4c814713b149108e488de855b025e4)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:35:59.789663 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b739cf0-857e-4a86-957b-3b11683a131c admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:35:59.800697 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d4e7b0-b397-43d3-8355-47fa30856d87 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:35:59.800950 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 899/1777] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Sep 6 04:35:59 2022] PUT /identity/v3/projects/b9576495e4e447d7b953b03118af9614/users/6445823c10e342b0bea67f61383ab645/roles/ea4c814713b149108e488de855b025e4 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 04:35:59.802180 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2884f001-1d2d-4b83-9a0b-9fe97a5628cf None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:35:59.802463 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2884f001-1d2d-4b83-9a0b-9fe97a5628cf None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:35:59.802734 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2884f001-1d2d-4b83-9a0b-9fe97a5628cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:35:59.810922 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d4e7b0-b397-43d3-8355-47fa30856d87 service cinder] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:35:59.820443 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 879/1778] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Sep 6 04:35:59 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 04:35:59.821295 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d4e7b0-b397-43d3-8355-47fa30856d87 service cinder] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:35:59.823903 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-2884f001-1d2d-4b83-9a0b-9fe97a5628cf None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:35:59.825110 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-2884f001-1d2d-4b83-9a0b-9fe97a5628cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:35:59.832949 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d4e7b0-b397-43d3-8355-47fa30856d87 service cinder] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:35:59.904173 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a541c9ca-e390-4f27-a76f-5dc02a62cdbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:35:59.904855 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a541c9ca-e390-4f27-a76f-5dc02a62cdbe admin admin] REQUEST_METHOD: `PUT` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:35:59.905139 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a541c9ca-e390-4f27-a76f-5dc02a62cdbe admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:35:59.905469 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a541c9ca-e390-4f27-a76f-5dc02a62cdbe admin admin] PATH_INFO: `/v3/projects/79cfd32b789a490e89b8e227ebd232ea/users/fe205f2b131646bb9355c578e5498027/roles/29607b2de39f40d68bb6ceef64b8a732` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:35:59.922467 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 900/1779] 10.222.0.30 () {58 vars in 958 bytes} [Tue Sep 6 04:35:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 04:35:59.923307 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a541c9ca-e390-4f27-a76f-5dc02a62cdbe admin admin] RBAC: Authorizing `identity:create_grant(project_id=79cfd32b789a490e89b8e227ebd232ea, user_id=fe205f2b131646bb9355c578e5498027, role_id=29607b2de39f40d68bb6ceef64b8a732)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:35:59.923898 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8ed026-3bf7-4dd9-8675-7b2ea46281cb None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:35:59.924255 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8ed026-3bf7-4dd9-8675-7b2ea46281cb None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:35:59.924598 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8ed026-3bf7-4dd9-8675-7b2ea46281cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:35:59.926373 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a541c9ca-e390-4f27-a76f-5dc02a62cdbe admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:35:59.956531 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-be8ed026-3bf7-4dd9-8675-7b2ea46281cb None None] MFA Rules not processed for user `6445823c10e342b0bea67f61383ab645`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:35:59.957795 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-be8ed026-3bf7-4dd9-8675-7b2ea46281cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:35:59.961086 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:36:00.015611 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 880/1780] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Sep 6 04:35:59 2022] PUT /identity/v3/projects/79cfd32b789a490e89b8e227ebd232ea/users/fe205f2b131646bb9355c578e5498027/roles/29607b2de39f40d68bb6ceef64b8a732 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 04:36:00.016770 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a541c9ca-e390-4f27-a76f-5dc02a62cdbe admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:36:00.043434 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 901/1781] 10.222.0.30 () {58 vars in 958 bytes} [Tue Sep 6 04:35:59 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 04:36:00.045338 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e8ae1c-c554-4e78-9af2-49c1fa19b118 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:36:00.045338 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e8ae1c-c554-4e78-9af2-49c1fa19b118 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:36:00.046001 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e8ae1c-c554-4e78-9af2-49c1fa19b118 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:36:00.086348 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-09e8ae1c-c554-4e78-9af2-49c1fa19b118 None None] MFA Rules not processed for user `fe205f2b131646bb9355c578e5498027`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:36:00.087727 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-09e8ae1c-c554-4e78-9af2-49c1fa19b118 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:36:00.098292 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a541c9ca-e390-4f27-a76f-5dc02a62cdbe admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:36:00.106039 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9d0d25-65e5-4873-9aac-fd3a19658a5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:36:00.107007 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9d0d25-65e5-4873-9aac-fd3a19658a5a admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:36:00.108103 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9d0d25-65e5-4873-9aac-fd3a19658a5a admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:36:00.108407 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9d0d25-65e5-4873-9aac-fd3a19658a5a admin admin] PATH_INFO: `/v3/domains` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:36:00.109086 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9d0d25-65e5-4873-9aac-fd3a19658a5a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:36:00.109427 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9d0d25-65e5-4873-9aac-fd3a19658a5a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:36:00.111140 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9d0d25-65e5-4873-9aac-fd3a19658a5a admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:36:00.124314 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 881/1782] 10.222.0.30 () {58 vars in 1153 bytes} [Tue Sep 6 04:36:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:36:00.125432 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9d0d25-65e5-4873-9aac-fd3a19658a5a admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:36:00.127078 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-5c9d0d25-65e5-4873-9aac-fd3a19658a5a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:36:00.187149 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 902/1783] 10.222.0.30 () {58 vars in 958 bytes} [Tue Sep 6 04:36:00 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 04:36:00.191219 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e8ae1c-c554-4e78-9af2-49c1fa19b118 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:36:00.210714 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e8ae1c-c554-4e78-9af2-49c1fa19b118 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:36:00.212000 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9d0d25-65e5-4873-9aac-fd3a19658a5a admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:36:00.216557 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fba38ff-2aa0-47b9-867e-1f4ce419b5eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:36:00.217207 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fba38ff-2aa0-47b9-867e-1f4ce419b5eb admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:36:00.217549 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fba38ff-2aa0-47b9-867e-1f4ce419b5eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:36:00.217645 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98c24f6-28e9-494d-8ffa-6501052903f1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd09b5ab675634a2eb4b03755743f7d0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b707733ce65c42edbc1fd95ad0a1c06c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:36:00.217925 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fba38ff-2aa0-47b9-867e-1f4ce419b5eb admin admin] PATH_INFO: `/v3/projects` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:36:00.218236 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98c24f6-28e9-494d-8ffa-6501052903f1 service cinder] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:36:00.218521 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98c24f6-28e9-494d-8ffa-6501052903f1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:36:00.218609 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fba38ff-2aa0-47b9-867e-1f4ce419b5eb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:36:00.218800 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98c24f6-28e9-494d-8ffa-6501052903f1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:36:00.220438 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fba38ff-2aa0-47b9-867e-1f4ce419b5eb admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:36:00.286411 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e98c24f6-28e9-494d-8ffa-6501052903f1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:36:00.298615 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e98c24f6-28e9-494d-8ffa-6501052903f1 service cinder] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:36:00.299499 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 882/1784] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Sep 6 04:36:00 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 04:36:00.301718 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98c24f6-28e9-494d-8ffa-6501052903f1 service cinder] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:36:00.328232 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98c24f6-28e9-494d-8ffa-6501052903f1 service cinder] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:36:00.334512 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b10b8c7-ea36-4072-8465-0cfbc53b3f0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:36:00.335233 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b10b8c7-ea36-4072-8465-0cfbc53b3f0b admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:36:00.335620 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b10b8c7-ea36-4072-8465-0cfbc53b3f0b admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:36:00.335996 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b10b8c7-ea36-4072-8465-0cfbc53b3f0b admin admin] PATH_INFO: `/v3/projects` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:36:00.336939 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b10b8c7-ea36-4072-8465-0cfbc53b3f0b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:36:00.339876 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b10b8c7-ea36-4072-8465-0cfbc53b3f0b admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:36:00.501717 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 903/1785] 10.222.0.30 () {60 vars in 1153 bytes} [Tue Sep 6 04:36:00 2022] POST /identity/v3/projects => generated 379 bytes in 313 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 04:36:00.507845 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fba38ff-2aa0-47b9-867e-1f4ce419b5eb admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:36:00.537260 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 883/1786] 10.222.0.30 () {60 vars in 1153 bytes} [Tue Sep 6 04:36:00 2022] POST /identity/v3/projects => generated 365 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 04:36:00.537260 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b10b8c7-ea36-4072-8465-0cfbc53b3f0b admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:36:00.580824 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fba38ff-2aa0-47b9-867e-1f4ce419b5eb admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:36:00.592022 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:36:00.600269 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b10b8c7-ea36-4072-8465-0cfbc53b3f0b admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:36:00.605458 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de46407-8e56-49db-a555-262eb3d0a471 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:36:00.606102 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de46407-8e56-49db-a555-262eb3d0a471 admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:36:00.606384 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de46407-8e56-49db-a555-262eb3d0a471 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:36:00.606667 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de46407-8e56-49db-a555-262eb3d0a471 admin admin] PATH_INFO: `/v3/users` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:36:00.607312 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de46407-8e56-49db-a555-262eb3d0a471 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:36:00.609231 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de46407-8e56-49db-a555-262eb3d0a471 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:36:00.638597 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70c8d1e-f522-4a96-bfab-ef779754c56a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:36:00.639208 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70c8d1e-f522-4a96-bfab-ef779754c56a admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:36:00.639484 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70c8d1e-f522-4a96-bfab-ef779754c56a admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:36:00.639749 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70c8d1e-f522-4a96-bfab-ef779754c56a admin admin] PATH_INFO: `/v3/users` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:36:00.640380 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70c8d1e-f522-4a96-bfab-ef779754c56a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:36:00.642359 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70c8d1e-f522-4a96-bfab-ef779754c56a admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:36:00.745205 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 884/1787] 10.222.0.30 () {60 vars in 1144 bytes} [Tue Sep 6 04:36:00 2022] POST /identity/v3/users => generated 289 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 04:36:00.752272 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 904/1788] 10.222.0.30 () {60 vars in 1144 bytes} [Tue Sep 6 04:36:00 2022] POST /identity/v3/users => generated 290 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 04:36:00.752387 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de46407-8e56-49db-a555-262eb3d0a471 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:36:00.754961 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70c8d1e-f522-4a96-bfab-ef779754c56a admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:36:00.765930 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70c8d1e-f522-4a96-bfab-ef779754c56a admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:36:00.771276 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d69cef7-028e-4759-92ac-4a206a9a17c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:36:00.771981 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d69cef7-028e-4759-92ac-4a206a9a17c4 admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:36:00.772330 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d69cef7-028e-4759-92ac-4a206a9a17c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:36:00.772677 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d69cef7-028e-4759-92ac-4a206a9a17c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:36:00.773424 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d69cef7-028e-4759-92ac-4a206a9a17c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:36:00.775384 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d69cef7-028e-4759-92ac-4a206a9a17c4 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:36:00.786930 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 905/1789] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Sep 6 04:36:00 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 06 04:36:00.788890 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d69cef7-028e-4759-92ac-4a206a9a17c4 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:36:00.801625 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d69cef7-028e-4759-92ac-4a206a9a17c4 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:36:00.806416 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd8a716-a31b-4fe9-8af0-4b7a59b0b583 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:36:00.807081 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd8a716-a31b-4fe9-8af0-4b7a59b0b583 admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:36:00.807368 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd8a716-a31b-4fe9-8af0-4b7a59b0b583 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:36:00.807646 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd8a716-a31b-4fe9-8af0-4b7a59b0b583 admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:36:00.808292 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd8a716-a31b-4fe9-8af0-4b7a59b0b583 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:36:00.809065 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de46407-8e56-49db-a555-262eb3d0a471 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:36:00.809941 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd8a716-a31b-4fe9-8af0-4b7a59b0b583 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:36:00.813015 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ea11ba-0961-4252-852a-80b75838bfe8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd09b5ab675634a2eb4b03755743f7d0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b707733ce65c42edbc1fd95ad0a1c06c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:36:00.813663 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ea11ba-0961-4252-852a-80b75838bfe8 service cinder] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:36:00.821339 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ea11ba-0961-4252-852a-80b75838bfe8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:36:00.821339 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ea11ba-0961-4252-852a-80b75838bfe8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:36:00.821339 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-b2ea11ba-0961-4252-852a-80b75838bfe8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:36:00.821594 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 906/1790] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Sep 6 04:36:00 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 04:36:00.821980 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd8a716-a31b-4fe9-8af0-4b7a59b0b583 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:36:00.832746 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd8a716-a31b-4fe9-8af0-4b7a59b0b583 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:36:00.839240 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf68f01e-3e2a-4ae2-b4be-3067ff2abc99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:36:00.840132 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf68f01e-3e2a-4ae2-b4be-3067ff2abc99 admin admin] REQUEST_METHOD: `PUT` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:36:00.840411 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf68f01e-3e2a-4ae2-b4be-3067ff2abc99 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:36:00.840678 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf68f01e-3e2a-4ae2-b4be-3067ff2abc99 admin admin] PATH_INFO: `/v3/projects/dd61b305100546eaa11b26e4a7e5054f/users/96215de3e4ed48968908a5ee5d7d13ae/roles/ea4c814713b149108e488de855b025e4` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:36:00.855529 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf68f01e-3e2a-4ae2-b4be-3067ff2abc99 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dd61b305100546eaa11b26e4a7e5054f, user_id=96215de3e4ed48968908a5ee5d7d13ae, role_id=ea4c814713b149108e488de855b025e4)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:36:00.858541 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf68f01e-3e2a-4ae2-b4be-3067ff2abc99 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:36:00.874673 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ea11ba-0961-4252-852a-80b75838bfe8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:36:00.876116 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ea11ba-0961-4252-852a-80b75838bfe8 service cinder] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:36:00.885055 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 885/1791] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Sep 6 04:36:00 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 04:36:00.886791 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ea11ba-0961-4252-852a-80b75838bfe8 service cinder] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:36:00.897200 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ea11ba-0961-4252-852a-80b75838bfe8 service cinder] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:36:00.902493 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17909af9-13bb-41b6-a4ab-df455875981e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:36:00.903112 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17909af9-13bb-41b6-a4ab-df455875981e admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:36:00.903427 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17909af9-13bb-41b6-a4ab-df455875981e admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:36:00.903705 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17909af9-13bb-41b6-a4ab-df455875981e admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:36:00.904349 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17909af9-13bb-41b6-a4ab-df455875981e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:36:00.905990 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17909af9-13bb-41b6-a4ab-df455875981e admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:36:00.916597 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 886/1792] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Sep 6 04:36:00 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 04:36:00.920627 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17909af9-13bb-41b6-a4ab-df455875981e admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:36:00.930753 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17909af9-13bb-41b6-a4ab-df455875981e admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:36:00.936722 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463d9990-d6ae-4fd5-b4d4-ea3f5e360b29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:36:00.937490 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463d9990-d6ae-4fd5-b4d4-ea3f5e360b29 admin admin] REQUEST_METHOD: `PUT` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:36:00.938084 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463d9990-d6ae-4fd5-b4d4-ea3f5e360b29 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:36:00.938361 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463d9990-d6ae-4fd5-b4d4-ea3f5e360b29 admin admin] PATH_INFO: `/v3/projects/77e7e8defac94cbd9457411dff1fd730/users/fb986b7722324c7fb15293580c87f43d/roles/29607b2de39f40d68bb6ceef64b8a732` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:36:00.945037 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 907/1793] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Sep 6 04:36:00 2022] PUT /identity/v3/projects/dd61b305100546eaa11b26e4a7e5054f/users/96215de3e4ed48968908a5ee5d7d13ae/roles/ea4c814713b149108e488de855b025e4 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 04:36:00.951058 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf68f01e-3e2a-4ae2-b4be-3067ff2abc99 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:36:00.954766 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463d9990-d6ae-4fd5-b4d4-ea3f5e360b29 admin admin] RBAC: Authorizing `identity:create_grant(project_id=77e7e8defac94cbd9457411dff1fd730, user_id=fb986b7722324c7fb15293580c87f43d, role_id=29607b2de39f40d68bb6ceef64b8a732)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:36:00.957521 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463d9990-d6ae-4fd5-b4d4-ea3f5e360b29 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:36:01.014001 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf68f01e-3e2a-4ae2-b4be-3067ff2abc99 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:36:01.018749 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1939c3c0-574d-487b-8f44-4b7fb152a5d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:36:01.019356 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1939c3c0-574d-487b-8f44-4b7fb152a5d2 admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:36:01.019634 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1939c3c0-574d-487b-8f44-4b7fb152a5d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:36:01.019911 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1939c3c0-574d-487b-8f44-4b7fb152a5d2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:36:01.020535 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1939c3c0-574d-487b-8f44-4b7fb152a5d2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:36:01.022177 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1939c3c0-574d-487b-8f44-4b7fb152a5d2 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:36:01.032494 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 908/1794] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Sep 6 04:36:00 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 06 04:36:01.036774 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1939c3c0-574d-487b-8f44-4b7fb152a5d2 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:36:01.047327 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1939c3c0-574d-487b-8f44-4b7fb152a5d2 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:36:01.049407 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 887/1795] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Sep 6 04:36:00 2022] PUT /identity/v3/projects/77e7e8defac94cbd9457411dff1fd730/users/fb986b7722324c7fb15293580c87f43d/roles/29607b2de39f40d68bb6ceef64b8a732 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 04:36:01.062628 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea49399-0ccf-40e4-a0d1-4a1fe683c56c None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:36:01.062921 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea49399-0ccf-40e4-a0d1-4a1fe683c56c None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:36:01.063212 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea49399-0ccf-40e4-a0d1-4a1fe683c56c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:36:01.096644 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-dea49399-0ccf-40e4-a0d1-4a1fe683c56c None None] MFA Rules not processed for user `fb986b7722324c7fb15293580c87f43d`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:36:01.097919 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-dea49399-0ccf-40e4-a0d1-4a1fe683c56c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:36:01.137834 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a73786-cd15-4cb4-b90e-4a26df62e1de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:36:01.138610 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a73786-cd15-4cb4-b90e-4a26df62e1de admin admin] REQUEST_METHOD: `PUT` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:36:01.138892 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a73786-cd15-4cb4-b90e-4a26df62e1de admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:36:01.139171 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a73786-cd15-4cb4-b90e-4a26df62e1de admin admin] PATH_INFO: `/v3/projects/dd61b305100546eaa11b26e4a7e5054f/users/96215de3e4ed48968908a5ee5d7d13ae/roles/ea4c814713b149108e488de855b025e4` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:36:01.141404 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a73786-cd15-4cb4-b90e-4a26df62e1de admin admin] RBAC: Authorizing `identity:create_grant(project_id=dd61b305100546eaa11b26e4a7e5054f, user_id=96215de3e4ed48968908a5ee5d7d13ae, role_id=ea4c814713b149108e488de855b025e4)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:36:01.144002 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a73786-cd15-4cb4-b90e-4a26df62e1de admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:36:01.156945 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 909/1796] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Sep 6 04:36:01 2022] PUT /identity/v3/projects/dd61b305100546eaa11b26e4a7e5054f/users/96215de3e4ed48968908a5ee5d7d13ae/roles/ea4c814713b149108e488de855b025e4 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 04:36:01.167719 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ed5d09-bc3e-477f-ad73-6759c4545120 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:36:01.171744 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ed5d09-bc3e-477f-ad73-6759c4545120 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:36:01.171861 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 888/1797] 10.222.0.30 () {58 vars in 958 bytes} [Tue Sep 6 04:36:01 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 04:36:01.171945 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ed5d09-bc3e-477f-ad73-6759c4545120 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:36:01.173093 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea49399-0ccf-40e4-a0d1-4a1fe683c56c None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:36:01.185427 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea49399-0ccf-40e4-a0d1-4a1fe683c56c None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:36:01.203687 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-32ed5d09-bc3e-477f-ad73-6759c4545120 None None] MFA Rules not processed for user `96215de3e4ed48968908a5ee5d7d13ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:36:01.205148 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-32ed5d09-bc3e-477f-ad73-6759c4545120 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:36:01.220177 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:36:01.266192 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333310ac-cb69-4df8-9c8a-2f605c675155 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:36:01.266192 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333310ac-cb69-4df8-9c8a-2f605c675155 admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:36:01.266192 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333310ac-cb69-4df8-9c8a-2f605c675155 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:36:01.266192 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333310ac-cb69-4df8-9c8a-2f605c675155 admin admin] PATH_INFO: `/v3/projects` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:36:01.266558 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333310ac-cb69-4df8-9c8a-2f605c675155 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:36:01.266558 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333310ac-cb69-4df8-9c8a-2f605c675155 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:36:01.315887 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 910/1798] 10.222.0.30 () {58 vars in 958 bytes} [Tue Sep 6 04:36:01 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 04:36:01.327990 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ed5d09-bc3e-477f-ad73-6759c4545120 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:36:01.373616 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 889/1799] 10.222.0.30 () {60 vars in 1153 bytes} [Tue Sep 6 04:36:01 2022] POST /identity/v3/projects => generated 377 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 04:36:01.384549 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333310ac-cb69-4df8-9c8a-2f605c675155 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:36:01.405236 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ed5d09-bc3e-477f-ad73-6759c4545120 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:36:01.450998 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333310ac-cb69-4df8-9c8a-2f605c675155 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:36:01.459069 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9205ba5-cbf9-4447-bf51-12500b4eca11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:36:01.459777 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9205ba5-cbf9-4447-bf51-12500b4eca11 admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:36:01.460145 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9205ba5-cbf9-4447-bf51-12500b4eca11 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:36:01.460445 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9205ba5-cbf9-4447-bf51-12500b4eca11 admin admin] PATH_INFO: `/v3/users` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:36:01.461261 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9205ba5-cbf9-4447-bf51-12500b4eca11 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:36:01.463994 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9205ba5-cbf9-4447-bf51-12500b4eca11 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:36:01.478664 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac28397-310e-4168-a4de-e5005e9d390b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd09b5ab675634a2eb4b03755743f7d0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b707733ce65c42edbc1fd95ad0a1c06c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:36:01.479909 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac28397-310e-4168-a4de-e5005e9d390b service cinder] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:36:01.480297 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac28397-310e-4168-a4de-e5005e9d390b service cinder] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:36:01.480642 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac28397-310e-4168-a4de-e5005e9d390b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:36:01.551049 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac28397-310e-4168-a4de-e5005e9d390b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:36:01.552524 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac28397-310e-4168-a4de-e5005e9d390b service cinder] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:36:01.562982 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 890/1800] 10.222.0.30 () {60 vars in 1144 bytes} [Tue Sep 6 04:36:01 2022] POST /identity/v3/users => generated 289 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 04:36:01.567618 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9205ba5-cbf9-4447-bf51-12500b4eca11 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:36:01.571697 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 911/1801] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Sep 6 04:36:01 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 244 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 06 04:36:01.576940 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9205ba5-cbf9-4447-bf51-12500b4eca11 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:36:01.582139 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4b1d72-1711-4456-b233-074cc5c9c288 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:36:01.583452 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4b1d72-1711-4456-b233-074cc5c9c288 admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:36:01.583452 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4b1d72-1711-4456-b233-074cc5c9c288 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:36:01.583452 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4b1d72-1711-4456-b233-074cc5c9c288 admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:36:01.584340 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4b1d72-1711-4456-b233-074cc5c9c288 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:36:01.585950 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4b1d72-1711-4456-b233-074cc5c9c288 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:36:01.600342 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 891/1802] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Sep 6 04:36:01 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 06 04:36:01.604700 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac28397-310e-4168-a4de-e5005e9d390b service cinder] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:36:01.615714 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac28397-310e-4168-a4de-e5005e9d390b service cinder] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:36:01.621203 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9a5cd7-6c81-46fd-9d5d-5e54295cfc86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:36:01.625182 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9a5cd7-6c81-46fd-9d5d-5e54295cfc86 admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:36:01.625182 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9a5cd7-6c81-46fd-9d5d-5e54295cfc86 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:36:01.625182 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9a5cd7-6c81-46fd-9d5d-5e54295cfc86 admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:36:01.625182 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa9a5cd7-6c81-46fd-9d5d-5e54295cfc86 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:36:01.625182 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa9a5cd7-6c81-46fd-9d5d-5e54295cfc86 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:36:01.635867 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 912/1803] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Sep 6 04:36:01 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 04:36:01.640926 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4b1d72-1711-4456-b233-074cc5c9c288 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:36:01.651651 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4b1d72-1711-4456-b233-074cc5c9c288 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:36:01.656835 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19609fcb-75fc-4720-a157-d1b0af05d313 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:36:01.657537 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19609fcb-75fc-4720-a157-d1b0af05d313 admin admin] REQUEST_METHOD: `PUT` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:36:01.657866 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19609fcb-75fc-4720-a157-d1b0af05d313 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:36:01.658172 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19609fcb-75fc-4720-a157-d1b0af05d313 admin admin] PATH_INFO: `/v3/projects/850d080e1c204836bbaec7e8b50a7f1e/users/0b25b2fd399f4778acc1727b802c7678/roles/29607b2de39f40d68bb6ceef64b8a732` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:36:01.673174 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19609fcb-75fc-4720-a157-d1b0af05d313 admin admin] RBAC: Authorizing `identity:create_grant(project_id=850d080e1c204836bbaec7e8b50a7f1e, user_id=0b25b2fd399f4778acc1727b802c7678, role_id=29607b2de39f40d68bb6ceef64b8a732)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:36:01.675815 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19609fcb-75fc-4720-a157-d1b0af05d313 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:36:01.746886 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 892/1804] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Sep 6 04:36:01 2022] PUT /identity/v3/projects/850d080e1c204836bbaec7e8b50a7f1e/users/0b25b2fd399f4778acc1727b802c7678/roles/29607b2de39f40d68bb6ceef64b8a732 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 04:36:01.754681 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a200deec-dd48-43b5-9cea-9c4c828a1141 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:36:01.754975 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a200deec-dd48-43b5-9cea-9c4c828a1141 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:36:01.755254 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a200deec-dd48-43b5-9cea-9c4c828a1141 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:36:01.784380 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-a200deec-dd48-43b5-9cea-9c4c828a1141 None None] MFA Rules not processed for user `0b25b2fd399f4778acc1727b802c7678`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:36:01.785630 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-a200deec-dd48-43b5-9cea-9c4c828a1141 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:36:01.823655 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19609fcb-75fc-4720-a157-d1b0af05d313 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:36:01.856607 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 913/1805] 10.222.0.30 () {58 vars in 958 bytes} [Tue Sep 6 04:36:01 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 04:36:01.860837 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a200deec-dd48-43b5-9cea-9c4c828a1141 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:36:01.879811 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19609fcb-75fc-4720-a157-d1b0af05d313 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:36:01.883468 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b3ea9a-339a-475a-b0de-568a72ea769b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd09b5ab675634a2eb4b03755743f7d0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b707733ce65c42edbc1fd95ad0a1c06c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:36:01.884072 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b3ea9a-339a-475a-b0de-568a72ea769b service cinder] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:36:01.884346 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b3ea9a-339a-475a-b0de-568a72ea769b service cinder] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:36:01.884616 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b3ea9a-339a-475a-b0de-568a72ea769b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:36:01.926658 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a200deec-dd48-43b5-9cea-9c4c828a1141 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:36:01.932552 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3248121-c203-428a-8ba2-377e52851503 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:36:01.933417 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3248121-c203-428a-8ba2-377e52851503 admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:36:01.934097 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3248121-c203-428a-8ba2-377e52851503 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:36:01.934384 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3248121-c203-428a-8ba2-377e52851503 admin admin] PATH_INFO: `/v3/projects` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:36:01.935303 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3248121-c203-428a-8ba2-377e52851503 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:36:01.939289 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3248121-c203-428a-8ba2-377e52851503 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:36:01.946642 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b3ea9a-339a-475a-b0de-568a72ea769b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:36:01.948498 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b3ea9a-339a-475a-b0de-568a72ea769b service cinder] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:36:01.959479 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 893/1806] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Sep 6 04:36:01 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 06 04:36:01.979966 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:DELETE=8,keystone:SELECT=88,keystone:INSERT=4 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:36:02.010474 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 914/1807] 10.222.0.30 () {60 vars in 1153 bytes} [Tue Sep 6 04:36:01 2022] POST /identity/v3/projects => generated 377 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 04:36:02.014053 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b3ea9a-339a-475a-b0de-568a72ea769b service cinder] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:36:02.024716 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b3ea9a-339a-475a-b0de-568a72ea769b service cinder] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:36:02.084798 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791d74b8-c2a3-44b2-a121-f314de7b52a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:36:02.084798 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791d74b8-c2a3-44b2-a121-f314de7b52a4 admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:36:02.084798 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791d74b8-c2a3-44b2-a121-f314de7b52a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:36:02.084798 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791d74b8-c2a3-44b2-a121-f314de7b52a4 admin admin] PATH_INFO: `/v3/users` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:36:02.085164 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791d74b8-c2a3-44b2-a121-f314de7b52a4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:36:02.086347 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791d74b8-c2a3-44b2-a121-f314de7b52a4 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:36:02.239626 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 894/1808] 10.222.0.30 () {60 vars in 1144 bytes} [Tue Sep 6 04:36:02 2022] POST /identity/v3/users => generated 295 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 04:36:02.240775 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3248121-c203-428a-8ba2-377e52851503 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:36:02.251447 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3248121-c203-428a-8ba2-377e52851503 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:36:02.256394 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637e2634-be50-44f2-8466-ee9b74d425e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:36:02.257036 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637e2634-be50-44f2-8466-ee9b74d425e4 admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:36:02.257332 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637e2634-be50-44f2-8466-ee9b74d425e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:36:02.257625 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637e2634-be50-44f2-8466-ee9b74d425e4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:36:02.258274 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637e2634-be50-44f2-8466-ee9b74d425e4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:36:02.259882 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637e2634-be50-44f2-8466-ee9b74d425e4 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:36:02.275197 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 915/1809] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Sep 6 04:36:02 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 04:36:02.277567 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791d74b8-c2a3-44b2-a121-f314de7b52a4 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:36:02.291825 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791d74b8-c2a3-44b2-a121-f314de7b52a4 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:36:02.296798 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4402f3a5-0c9b-426d-b35d-6fbc4e9a87a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:36:02.297665 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4402f3a5-0c9b-426d-b35d-6fbc4e9a87a5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:36:02.297791 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4402f3a5-0c9b-426d-b35d-6fbc4e9a87a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:36:02.298045 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4402f3a5-0c9b-426d-b35d-6fbc4e9a87a5 admin admin] PATH_INFO: `/v3/projects/fffff36fe18b457a910f7447db81ccc5/users/1a57fbc71cac44449c48e783ad17054c/roles/ea4c814713b149108e488de855b025e4` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:36:02.313929 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4402f3a5-0c9b-426d-b35d-6fbc4e9a87a5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fffff36fe18b457a910f7447db81ccc5, user_id=1a57fbc71cac44449c48e783ad17054c, role_id=ea4c814713b149108e488de855b025e4)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:36:02.316713 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4402f3a5-0c9b-426d-b35d-6fbc4e9a87a5 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:36:02.403211 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 895/1810] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Sep 6 04:36:02 2022] PUT /identity/v3/projects/fffff36fe18b457a910f7447db81ccc5/users/1a57fbc71cac44449c48e783ad17054c/roles/ea4c814713b149108e488de855b025e4 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 04:36:02.413477 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637e2634-be50-44f2-8466-ee9b74d425e4 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:36:02.417482 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637e2634-be50-44f2-8466-ee9b74d425e4 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:36:02.493776 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:36:02.503731 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7cd39f-24a3-40f1-acf6-72f2abc715a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:36:02.503731 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7cd39f-24a3-40f1-acf6-72f2abc715a5 admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:36:02.503731 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7cd39f-24a3-40f1-acf6-72f2abc715a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:36:02.503731 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7cd39f-24a3-40f1-acf6-72f2abc715a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:36:02.504538 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7cd39f-24a3-40f1-acf6-72f2abc715a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:36:02.506221 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7cd39f-24a3-40f1-acf6-72f2abc715a5 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:36:02.523842 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 916/1811] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Sep 6 04:36:02 2022] GET /identity/v3/roles => generated 1420 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 04:36:02.525131 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4402f3a5-0c9b-426d-b35d-6fbc4e9a87a5 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:36:02.538757 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4402f3a5-0c9b-426d-b35d-6fbc4e9a87a5 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:36:02.544569 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22abd15b-cb28-40e9-87d8-2321247526ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:36:02.545243 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22abd15b-cb28-40e9-87d8-2321247526ef admin admin] REQUEST_METHOD: `PUT` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:36:02.545557 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22abd15b-cb28-40e9-87d8-2321247526ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:36:02.546467 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22abd15b-cb28-40e9-87d8-2321247526ef admin admin] PATH_INFO: `/v3/projects/fffff36fe18b457a910f7447db81ccc5/users/1a57fbc71cac44449c48e783ad17054c/roles/ea4c814713b149108e488de855b025e4` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:36:02.548427 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22abd15b-cb28-40e9-87d8-2321247526ef admin admin] RBAC: Authorizing `identity:create_grant(project_id=fffff36fe18b457a910f7447db81ccc5, user_id=1a57fbc71cac44449c48e783ad17054c, role_id=ea4c814713b149108e488de855b025e4)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:36:02.551479 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22abd15b-cb28-40e9-87d8-2321247526ef admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:36:02.565372 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 896/1812] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Sep 6 04:36:02 2022] PUT /identity/v3/projects/fffff36fe18b457a910f7447db81ccc5/users/1a57fbc71cac44449c48e783ad17054c/roles/ea4c814713b149108e488de855b025e4 => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 04:36:02.575669 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7038a1b-ed4c-43b7-b79a-3bcddc5b722a None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:36:02.575955 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7038a1b-ed4c-43b7-b79a-3bcddc5b722a None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:36:02.576232 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7038a1b-ed4c-43b7-b79a-3bcddc5b722a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:36:02.619903 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-a7038a1b-ed4c-43b7-b79a-3bcddc5b722a None None] MFA Rules not processed for user `1a57fbc71cac44449c48e783ad17054c`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:36:02.623787 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-a7038a1b-ed4c-43b7-b79a-3bcddc5b722a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:36:02.709991 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 917/1813] 10.222.0.30 () {58 vars in 958 bytes} [Tue Sep 6 04:36:02 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 04:36:02.765487 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22abd15b-cb28-40e9-87d8-2321247526ef admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:36:02.822996 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22abd15b-cb28-40e9-87d8-2321247526ef admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:36:02.833658 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97c09e6-33ae-4655-9e85-e02ced5cb626 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd09b5ab675634a2eb4b03755743f7d0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b707733ce65c42edbc1fd95ad0a1c06c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:36:02.835124 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97c09e6-33ae-4655-9e85-e02ced5cb626 service cinder] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:36:02.835630 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97c09e6-33ae-4655-9e85-e02ced5cb626 service cinder] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:36:02.836104 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97c09e6-33ae-4655-9e85-e02ced5cb626 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:36:02.849690 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97c09e6-33ae-4655-9e85-e02ced5cb626 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:36:02.851164 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97c09e6-33ae-4655-9e85-e02ced5cb626 service cinder] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:36:02.863021 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 897/1814] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Sep 6 04:36:02 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 04:36:03.012399 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7038a1b-ed4c-43b7-b79a-3bcddc5b722a None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:36:03.023013 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7038a1b-ed4c-43b7-b79a-3bcddc5b722a None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:36:03.027401 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb4e22a-72fe-4170-98f5-58774bf2a737 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd09b5ab675634a2eb4b03755743f7d0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b707733ce65c42edbc1fd95ad0a1c06c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:36:03.032494 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb4e22a-72fe-4170-98f5-58774bf2a737 service cinder] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:36:03.032494 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb4e22a-72fe-4170-98f5-58774bf2a737 service cinder] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:36:03.032494 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb4e22a-72fe-4170-98f5-58774bf2a737 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:36:03.084646 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb4e22a-72fe-4170-98f5-58774bf2a737 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:36:03.086112 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb4e22a-72fe-4170-98f5-58774bf2a737 service cinder] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:36:03.099712 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 918/1815] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Sep 6 04:36:03 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 06 04:36:03.814587 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:36:04.299452 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:36:05.697195 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=91,keystone:INSERT=4,keystone:DELETE=5 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:36:05.714738 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97c09e6-33ae-4655-9e85-e02ced5cb626 service cinder] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:36:05.716271 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-f97c09e6-33ae-4655-9e85-e02ced5cb626 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:36:05.768395 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:DELETE=7,keystone:SELECT=89,keystone:INSERT=4 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:36:05.784623 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97c09e6-33ae-4655-9e85-e02ced5cb626 service cinder] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:36:05.789313 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e712cf5-8730-48d5-bcaf-61fce6e29314 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:36:05.789963 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e712cf5-8730-48d5-bcaf-61fce6e29314 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:36:05.790246 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e712cf5-8730-48d5-bcaf-61fce6e29314 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:36:05.790521 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e712cf5-8730-48d5-bcaf-61fce6e29314 admin admin] PATH_INFO: `/v3/users/79fe4ce702654b9a957544d55e5df57c` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:36:05.791687 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e712cf5-8730-48d5-bcaf-61fce6e29314 admin admin] RBAC: Authorizing `identity:delete_user(user_id=79fe4ce702654b9a957544d55e5df57c)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:36:05.793748 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e712cf5-8730-48d5-bcaf-61fce6e29314 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:36:06.003239 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-2e712cf5-8730-48d5-bcaf-61fce6e29314 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '79fe4ce702654b9a957544d55e5df57c'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:36:06.106921 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-2e712cf5-8730-48d5-bcaf-61fce6e29314 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '79fe4ce702654b9a957544d55e5df57c'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:36:06.107684 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-2e712cf5-8730-48d5-bcaf-61fce6e29314 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '79fe4ce702654b9a957544d55e5df57c'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:36:06.122105 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-2e712cf5-8730-48d5-bcaf-61fce6e29314 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '79fe4ce702654b9a957544d55e5df57c'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:36:06.133339 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-2e712cf5-8730-48d5-bcaf-61fce6e29314 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '79fe4ce702654b9a957544d55e5df57c'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:36:06.141504 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 898/1816] 10.222.0.30 () {58 vars in 1224 bytes} [Tue Sep 6 04:36:05 2022] DELETE /identity/v3/users/79fe4ce702654b9a957544d55e5df57c => generated 0 bytes in 428 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 04:36:06.147697 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd3223f-b6cc-493c-b7a6-60d641b482ef None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:36:06.147697 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd3223f-b6cc-493c-b7a6-60d641b482ef None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:36:06.147697 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd3223f-b6cc-493c-b7a6-60d641b482ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:36:06.175296 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-1dd3223f-b6cc-493c-b7a6-60d641b482ef None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:36:06.176975 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-1dd3223f-b6cc-493c-b7a6-60d641b482ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:36:06.264286 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 919/1817] 10.222.0.30 () {58 vars in 958 bytes} [Tue Sep 6 04:36:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 04:36:06.274654 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e712cf5-8730-48d5-bcaf-61fce6e29314 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:36:06.277063 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-2e712cf5-8730-48d5-bcaf-61fce6e29314 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:36:06.375369 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e712cf5-8730-48d5-bcaf-61fce6e29314 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:36:06.375369 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e2f660-2c0c-4b4a-8d4a-5b7be9f77e5c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90492440f6a6471e8392361c098428aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b707733ce65c42edbc1fd95ad0a1c06c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:36:06.376087 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e2f660-2c0c-4b4a-8d4a-5b7be9f77e5c service neutron] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:36:06.376470 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e2f660-2c0c-4b4a-8d4a-5b7be9f77e5c service neutron] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:36:06.376840 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e2f660-2c0c-4b4a-8d4a-5b7be9f77e5c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:36:06.381951 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd3223f-b6cc-493c-b7a6-60d641b482ef None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:36:06.383542 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-1dd3223f-b6cc-493c-b7a6-60d641b482ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:36:06.396737 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e2f660-2c0c-4b4a-8d4a-5b7be9f77e5c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:36:06.399065 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e2f660-2c0c-4b4a-8d4a-5b7be9f77e5c service neutron] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:36:06.401513 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd3223f-b6cc-493c-b7a6-60d641b482ef None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:36:06.407427 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c821e1d-4bbf-48e9-a239-155dedad4661 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:36:06.408264 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c821e1d-4bbf-48e9-a239-155dedad4661 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:36:06.412327 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c821e1d-4bbf-48e9-a239-155dedad4661 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:36:06.412327 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c821e1d-4bbf-48e9-a239-155dedad4661 admin admin] PATH_INFO: `/v3/users/fe205f2b131646bb9355c578e5498027` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:36:06.412327 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c821e1d-4bbf-48e9-a239-155dedad4661 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fe205f2b131646bb9355c578e5498027)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:36:06.412950 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c821e1d-4bbf-48e9-a239-155dedad4661 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:36:06.417804 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 899/1818] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Sep 6 04:36:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 04:36:06.526311 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a643b3-1007-4b06-a93d-40d37bd80c0e None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:36:06.527208 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a643b3-1007-4b06-a93d-40d37bd80c0e None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:36:06.527698 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a643b3-1007-4b06-a93d-40d37bd80c0e None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:36:06.529996 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 900/1819] 10.222.0.30 () {54 vars in 921 bytes} [Tue Sep 6 04:36:06 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 06 04:36:06.550335 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744e54e2-3427-4b9f-988f-5eca4500ca05 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:36:06.550818 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744e54e2-3427-4b9f-988f-5eca4500ca05 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:36:06.551273 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744e54e2-3427-4b9f-988f-5eca4500ca05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:36:06.615827 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-744e54e2-3427-4b9f-988f-5eca4500ca05 None None] MFA Rules not processed for user `9db138b77afa4b068244b8ef715404fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:36:06.617062 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-744e54e2-3427-4b9f-988f-5eca4500ca05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:36:06.693215 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 901/1820] 10.222.0.30 () {58 vars in 1018 bytes} [Tue Sep 6 04:36:06 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 04:36:06.719625 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744e54e2-3427-4b9f-988f-5eca4500ca05 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:36:06.721648 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-744e54e2-3427-4b9f-988f-5eca4500ca05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:36:06.722173 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-1c821e1d-4bbf-48e9-a239-155dedad4661 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fe205f2b131646bb9355c578e5498027'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:36:06.737786 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744e54e2-3427-4b9f-988f-5eca4500ca05 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:36:06.743894 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce283694-120b-440a-bc66-2aeaa64fec76 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9db138b77afa4b068244b8ef715404fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b707733ce65c42edbc1fd95ad0a1c06c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:36:06.746776 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce283694-120b-440a-bc66-2aeaa64fec76 service nova] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:36:06.747083 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce283694-120b-440a-bc66-2aeaa64fec76 service nova] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:36:06.747370 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce283694-120b-440a-bc66-2aeaa64fec76 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:36:06.758268 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce283694-120b-440a-bc66-2aeaa64fec76 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:36:06.760500 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce283694-120b-440a-bc66-2aeaa64fec76 service nova] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:36:06.773098 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 902/1821] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Sep 6 04:36:06 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 04:36:06.794443 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-1c821e1d-4bbf-48e9-a239-155dedad4661 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fe205f2b131646bb9355c578e5498027'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:36:06.795918 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-1c821e1d-4bbf-48e9-a239-155dedad4661 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fe205f2b131646bb9355c578e5498027'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:36:06.814750 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-1c821e1d-4bbf-48e9-a239-155dedad4661 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fe205f2b131646bb9355c578e5498027'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:36:06.822165 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-1c821e1d-4bbf-48e9-a239-155dedad4661 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fe205f2b131646bb9355c578e5498027'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:36:06.831396 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 920/1822] 10.222.0.30 () {58 vars in 1224 bytes} [Tue Sep 6 04:36:06 2022] DELETE /identity/v3/users/fe205f2b131646bb9355c578e5498027 => generated 0 bytes in 450 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 04:36:06.835962 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc0a3cc-e8f9-4815-9e1f-02e0cb93f27f None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:36:06.836255 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc0a3cc-e8f9-4815-9e1f-02e0cb93f27f None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:36:06.836537 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc0a3cc-e8f9-4815-9e1f-02e0cb93f27f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:36:06.856300 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-7cc0a3cc-e8f9-4815-9e1f-02e0cb93f27f None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:36:06.857652 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-7cc0a3cc-e8f9-4815-9e1f-02e0cb93f27f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:36:06.888028 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:DELETE=5,keystone:SELECT=93,keystone:INSERT=2 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:36:06.922046 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 903/1823] 10.222.0.30 () {58 vars in 958 bytes} [Tue Sep 6 04:36:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 04:36:06.932341 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c821e1d-4bbf-48e9-a239-155dedad4661 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:36:06.934244 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-1c821e1d-4bbf-48e9-a239-155dedad4661 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:36:06.945449 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:36:06.953114 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc0a3cc-e8f9-4815-9e1f-02e0cb93f27f None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:36:06.955813 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-7cc0a3cc-e8f9-4815-9e1f-02e0cb93f27f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:36:06.978315 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc0a3cc-e8f9-4815-9e1f-02e0cb93f27f None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:36:06.985603 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e1bf90-9fce-4ca9-8380-96309701ca7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:36:06.986273 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e1bf90-9fce-4ca9-8380-96309701ca7b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:36:06.986556 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e1bf90-9fce-4ca9-8380-96309701ca7b admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:36:06.986831 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e1bf90-9fce-4ca9-8380-96309701ca7b admin admin] PATH_INFO: `/v3/projects/422cae1387de42c9b2379a9ab95e586a` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:36:06.988011 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e1bf90-9fce-4ca9-8380-96309701ca7b admin admin] RBAC: Authorizing `identity:delete_project(project_id=422cae1387de42c9b2379a9ab95e586a)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:36:06.991337 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e1bf90-9fce-4ca9-8380-96309701ca7b admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:36:07.045251 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c821e1d-4bbf-48e9-a239-155dedad4661 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:36:07.049138 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14ef329-c27f-4356-99f0-0bccb8a77c94 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90492440f6a6471e8392361c098428aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b707733ce65c42edbc1fd95ad0a1c06c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:36:07.049854 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14ef329-c27f-4356-99f0-0bccb8a77c94 service neutron] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:36:07.050129 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14ef329-c27f-4356-99f0-0bccb8a77c94 service neutron] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:36:07.050404 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14ef329-c27f-4356-99f0-0bccb8a77c94 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:36:07.072327 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14ef329-c27f-4356-99f0-0bccb8a77c94 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:36:07.075598 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14ef329-c27f-4356-99f0-0bccb8a77c94 service neutron] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:36:07.089048 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 921/1824] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Sep 6 04:36:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 04:36:07.211215 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-e5e1bf90-9fce-4ca9-8380-96309701ca7b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '422cae1387de42c9b2379a9ab95e586a', 'request_id': 'req-e5e1bf90-9fce-4ca9-8380-96309701ca7b'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:36:07.333354 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-e5e1bf90-9fce-4ca9-8380-96309701ca7b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '422cae1387de42c9b2379a9ab95e586a', 'request_id': 'req-e5e1bf90-9fce-4ca9-8380-96309701ca7b'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:36:07.333354 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-e5e1bf90-9fce-4ca9-8380-96309701ca7b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '422cae1387de42c9b2379a9ab95e586a', 'request_id': 'req-e5e1bf90-9fce-4ca9-8380-96309701ca7b'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:36:07.346403 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-e5e1bf90-9fce-4ca9-8380-96309701ca7b admin admin] The token cache is being invalidate because project 422cae1387de42c9b2379a9ab95e586a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81697) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 04:36:07.346793 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-e5e1bf90-9fce-4ca9-8380-96309701ca7b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:36:07.347644 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-e5e1bf90-9fce-4ca9-8380-96309701ca7b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:36:07.349613 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 904/1825] 10.222.0.30 () {58 vars in 1233 bytes} [Tue Sep 6 04:36:06 2022] DELETE /identity/v3/projects/422cae1387de42c9b2379a9ab95e586a => generated 0 bytes in 397 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 04:36:07.358853 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14ef329-c27f-4356-99f0-0bccb8a77c94 service neutron] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:36:07.371156 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14ef329-c27f-4356-99f0-0bccb8a77c94 service neutron] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:36:07.445296 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883b779e-6b0b-4709-bb73-4cd8c49bbe6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:36:07.446110 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883b779e-6b0b-4709-bb73-4cd8c49bbe6c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:36:07.446655 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883b779e-6b0b-4709-bb73-4cd8c49bbe6c admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:36:07.446998 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883b779e-6b0b-4709-bb73-4cd8c49bbe6c admin admin] PATH_INFO: `/v3/users/00509315822d4e4380ea31abc5403343` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:36:07.450186 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883b779e-6b0b-4709-bb73-4cd8c49bbe6c admin admin] RBAC: Authorizing `identity:delete_user(user_id=00509315822d4e4380ea31abc5403343)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:36:07.452611 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883b779e-6b0b-4709-bb73-4cd8c49bbe6c admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:36:07.460134 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e1bf90-9fce-4ca9-8380-96309701ca7b admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:36:07.468424 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-e5e1bf90-9fce-4ca9-8380-96309701ca7b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:36:07.504188 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e1bf90-9fce-4ca9-8380-96309701ca7b admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:36:07.512625 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62bccaa3-c5e1-4f16-a3ff-ec6bf2b3fa49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:36:07.513490 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bccaa3-c5e1-4f16-a3ff-ec6bf2b3fa49 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:36:07.513843 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bccaa3-c5e1-4f16-a3ff-ec6bf2b3fa49 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:36:07.514515 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bccaa3-c5e1-4f16-a3ff-ec6bf2b3fa49 admin admin] PATH_INFO: `/v3/users/fb986b7722324c7fb15293580c87f43d` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:36:07.516517 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62bccaa3-c5e1-4f16-a3ff-ec6bf2b3fa49 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fb986b7722324c7fb15293580c87f43d)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:36:07.518621 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62bccaa3-c5e1-4f16-a3ff-ec6bf2b3fa49 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:36:07.681096 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-883b779e-6b0b-4709-bb73-4cd8c49bbe6c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '00509315822d4e4380ea31abc5403343'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:36:07.708638 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-62bccaa3-c5e1-4f16-a3ff-ec6bf2b3fa49 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fb986b7722324c7fb15293580c87f43d'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:36:07.803716 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-883b779e-6b0b-4709-bb73-4cd8c49bbe6c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '00509315822d4e4380ea31abc5403343'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:36:07.803716 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-883b779e-6b0b-4709-bb73-4cd8c49bbe6c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '00509315822d4e4380ea31abc5403343'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:36:07.823625 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-883b779e-6b0b-4709-bb73-4cd8c49bbe6c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '00509315822d4e4380ea31abc5403343'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:36:07.835607 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-883b779e-6b0b-4709-bb73-4cd8c49bbe6c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '00509315822d4e4380ea31abc5403343'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:36:07.843098 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 922/1826] 10.222.0.30 () {58 vars in 1224 bytes} [Tue Sep 6 04:36:07 2022] DELETE /identity/v3/users/00509315822d4e4380ea31abc5403343 => generated 0 bytes in 485 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 04:36:07.845110 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883b779e-6b0b-4709-bb73-4cd8c49bbe6c admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:36:07.846949 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-883b779e-6b0b-4709-bb73-4cd8c49bbe6c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:36:07.881705 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-62bccaa3-c5e1-4f16-a3ff-ec6bf2b3fa49 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fb986b7722324c7fb15293580c87f43d'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:36:07.882829 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-62bccaa3-c5e1-4f16-a3ff-ec6bf2b3fa49 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fb986b7722324c7fb15293580c87f43d'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:36:07.893897 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-62bccaa3-c5e1-4f16-a3ff-ec6bf2b3fa49 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fb986b7722324c7fb15293580c87f43d'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:36:07.926491 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-62bccaa3-c5e1-4f16-a3ff-ec6bf2b3fa49 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fb986b7722324c7fb15293580c87f43d'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:36:07.934190 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 905/1827] 10.222.0.30 () {58 vars in 1224 bytes} [Tue Sep 6 04:36:07 2022] DELETE /identity/v3/users/fb986b7722324c7fb15293580c87f43d => generated 0 bytes in 475 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 04:36:07.938795 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b200136e-63b0-489e-983f-c61733fcbf14 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:36:07.939242 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b200136e-63b0-489e-983f-c61733fcbf14 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:36:07.954592 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b200136e-63b0-489e-983f-c61733fcbf14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:36:07.954880 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883b779e-6b0b-4709-bb73-4cd8c49bbe6c admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:36:07.959616 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-b200136e-63b0-489e-983f-c61733fcbf14 None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:36:07.960850 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-b200136e-63b0-489e-983f-c61733fcbf14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:36:08.001900 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f66080-7ab1-445e-b119-f870f955bf3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:36:08.002534 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f66080-7ab1-445e-b119-f870f955bf3c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:36:08.002811 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f66080-7ab1-445e-b119-f870f955bf3c admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:36:08.003120 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f66080-7ab1-445e-b119-f870f955bf3c admin admin] PATH_INFO: `/v3/projects/79cfd32b789a490e89b8e227ebd232ea` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:36:08.004204 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f66080-7ab1-445e-b119-f870f955bf3c admin admin] RBAC: Authorizing `identity:delete_project(project_id=79cfd32b789a490e89b8e227ebd232ea)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:36:08.006144 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f66080-7ab1-445e-b119-f870f955bf3c admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:36:08.021838 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 906/1828] 10.222.0.30 () {58 vars in 958 bytes} [Tue Sep 6 04:36:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 04:36:08.034903 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b200136e-63b0-489e-983f-c61733fcbf14 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:36:08.034903 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-b200136e-63b0-489e-983f-c61733fcbf14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:36:08.096463 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b200136e-63b0-489e-983f-c61733fcbf14 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:36:08.101111 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84de294-314e-4cc7-8569-3312b1caa61a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90492440f6a6471e8392361c098428aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b707733ce65c42edbc1fd95ad0a1c06c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:36:08.104055 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84de294-314e-4cc7-8569-3312b1caa61a service neutron] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:36:08.104055 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84de294-314e-4cc7-8569-3312b1caa61a service neutron] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:36:08.104055 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84de294-314e-4cc7-8569-3312b1caa61a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:36:08.124895 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e84de294-314e-4cc7-8569-3312b1caa61a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:36:08.129356 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e84de294-314e-4cc7-8569-3312b1caa61a service neutron] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:36:08.136597 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-06f66080-7ab1-445e-b119-f870f955bf3c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '79cfd32b789a490e89b8e227ebd232ea', 'request_id': 'req-06f66080-7ab1-445e-b119-f870f955bf3c'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:36:08.150643 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-06f66080-7ab1-445e-b119-f870f955bf3c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '79cfd32b789a490e89b8e227ebd232ea', 'request_id': 'req-06f66080-7ab1-445e-b119-f870f955bf3c'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:36:08.154131 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 907/1829] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Sep 6 04:36:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 06 04:36:08.237516 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-06f66080-7ab1-445e-b119-f870f955bf3c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '79cfd32b789a490e89b8e227ebd232ea', 'request_id': 'req-06f66080-7ab1-445e-b119-f870f955bf3c'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:36:08.250881 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-06f66080-7ab1-445e-b119-f870f955bf3c admin admin] The token cache is being invalidate because project 79cfd32b789a490e89b8e227ebd232ea was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81698) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 04:36:08.251180 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-06f66080-7ab1-445e-b119-f870f955bf3c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:36:08.251975 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-06f66080-7ab1-445e-b119-f870f955bf3c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:36:08.253573 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 923/1830] 10.222.0.30 () {58 vars in 1233 bytes} [Tue Sep 6 04:36:07 2022] DELETE /identity/v3/projects/79cfd32b789a490e89b8e227ebd232ea => generated 0 bytes in 410 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 04:36:08.257794 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84de294-314e-4cc7-8569-3312b1caa61a service neutron] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:36:08.259297 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-e84de294-314e-4cc7-8569-3312b1caa61a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:36:08.273354 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84de294-314e-4cc7-8569-3312b1caa61a service neutron] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:36:08.331662 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415d7b56-63d5-4db0-9501-2bcead70fb98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:36:08.332739 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415d7b56-63d5-4db0-9501-2bcead70fb98 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:36:08.333056 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415d7b56-63d5-4db0-9501-2bcead70fb98 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:36:08.334104 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415d7b56-63d5-4db0-9501-2bcead70fb98 admin admin] PATH_INFO: `/v3/users/96215de3e4ed48968908a5ee5d7d13ae` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:36:08.335320 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415d7b56-63d5-4db0-9501-2bcead70fb98 admin admin] RBAC: Authorizing `identity:delete_user(user_id=96215de3e4ed48968908a5ee5d7d13ae)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:36:08.337614 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415d7b56-63d5-4db0-9501-2bcead70fb98 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:36:08.361570 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f66080-7ab1-445e-b119-f870f955bf3c admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:36:08.361570 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-06f66080-7ab1-445e-b119-f870f955bf3c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:36:08.383720 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f66080-7ab1-445e-b119-f870f955bf3c admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:36:08.403244 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea1222a-7b5e-493f-8bb3-604b6dff075f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:36:08.404138 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea1222a-7b5e-493f-8bb3-604b6dff075f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:36:08.404425 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea1222a-7b5e-493f-8bb3-604b6dff075f admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:36:08.404702 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea1222a-7b5e-493f-8bb3-604b6dff075f admin admin] PATH_INFO: `/v3/projects/20d9bb2cd54c4208a94d17548633f468` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:36:08.412332 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea1222a-7b5e-493f-8bb3-604b6dff075f admin admin] RBAC: Authorizing `identity:delete_project(project_id=20d9bb2cd54c4208a94d17548633f468)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:36:08.414939 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea1222a-7b5e-493f-8bb3-604b6dff075f admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:36:08.448451 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:36:08.546803 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:36:08.566901 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-7ea1222a-7b5e-493f-8bb3-604b6dff075f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '20d9bb2cd54c4208a94d17548633f468', 'request_id': 'req-7ea1222a-7b5e-493f-8bb3-604b6dff075f'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:36:08.570741 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-415d7b56-63d5-4db0-9501-2bcead70fb98 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '96215de3e4ed48968908a5ee5d7d13ae'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:36:08.582509 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-7ea1222a-7b5e-493f-8bb3-604b6dff075f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '20d9bb2cd54c4208a94d17548633f468', 'request_id': 'req-7ea1222a-7b5e-493f-8bb3-604b6dff075f'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:36:08.665093 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-415d7b56-63d5-4db0-9501-2bcead70fb98 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '96215de3e4ed48968908a5ee5d7d13ae'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:36:08.665775 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-415d7b56-63d5-4db0-9501-2bcead70fb98 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '96215de3e4ed48968908a5ee5d7d13ae'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:36:08.674643 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-415d7b56-63d5-4db0-9501-2bcead70fb98 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '96215de3e4ed48968908a5ee5d7d13ae'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:36:08.687142 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-415d7b56-63d5-4db0-9501-2bcead70fb98 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '96215de3e4ed48968908a5ee5d7d13ae'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:36:08.693343 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 908/1831] 10.222.0.30 () {58 vars in 1224 bytes} [Tue Sep 6 04:36:08 2022] DELETE /identity/v3/users/96215de3e4ed48968908a5ee5d7d13ae => generated 0 bytes in 436 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 04:36:08.750598 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-7ea1222a-7b5e-493f-8bb3-604b6dff075f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '20d9bb2cd54c4208a94d17548633f468', 'request_id': 'req-7ea1222a-7b5e-493f-8bb3-604b6dff075f'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:36:08.790259 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415d7b56-63d5-4db0-9501-2bcead70fb98 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:36:08.790259 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-415d7b56-63d5-4db0-9501-2bcead70fb98 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:36:08.790523 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-7ea1222a-7b5e-493f-8bb3-604b6dff075f admin admin] The token cache is being invalidate because project 20d9bb2cd54c4208a94d17548633f468 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81698) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 04:36:08.790523 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-7ea1222a-7b5e-493f-8bb3-604b6dff075f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:36:08.790523 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-7ea1222a-7b5e-493f-8bb3-604b6dff075f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:36:08.790523 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 924/1832] 10.222.0.30 () {58 vars in 1233 bytes} [Tue Sep 6 04:36:08 2022] DELETE /identity/v3/projects/20d9bb2cd54c4208a94d17548633f468 => generated 0 bytes in 414 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 04:36:08.790523 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a72b54-968e-4847-979c-82ca7eda25b0 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:36:08.790958 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a72b54-968e-4847-979c-82ca7eda25b0 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:36:08.790958 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a72b54-968e-4847-979c-82ca7eda25b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:36:08.832000 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-41a72b54-968e-4847-979c-82ca7eda25b0 None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:36:08.836363 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-41a72b54-968e-4847-979c-82ca7eda25b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:36:08.893926 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415d7b56-63d5-4db0-9501-2bcead70fb98 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:36:08.906530 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d32295-c061-4abc-ace8-863e68993fb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:36:08.909221 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d32295-c061-4abc-ace8-863e68993fb7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:36:08.909634 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d32295-c061-4abc-ace8-863e68993fb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:36:08.910032 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d32295-c061-4abc-ace8-863e68993fb7 admin admin] PATH_INFO: `/v3/projects/77e7e8defac94cbd9457411dff1fd730` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:36:08.913360 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d32295-c061-4abc-ace8-863e68993fb7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=77e7e8defac94cbd9457411dff1fd730)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:36:08.915311 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d32295-c061-4abc-ace8-863e68993fb7 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:36:08.962573 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 925/1833] 10.222.0.30 () {58 vars in 958 bytes} [Tue Sep 6 04:36:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 04:36:08.984658 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414d0f8c-7875-46bf-b77b-33698de0193d None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:36:08.984658 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414d0f8c-7875-46bf-b77b-33698de0193d None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:36:08.985052 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414d0f8c-7875-46bf-b77b-33698de0193d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:36:09.054016 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-414d0f8c-7875-46bf-b77b-33698de0193d None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:36:09.055309 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-414d0f8c-7875-46bf-b77b-33698de0193d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:36:09.074605 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 926/1834] 10.222.0.30 () {58 vars in 958 bytes} [Tue Sep 6 04:36:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 04:36:09.074605 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414d0f8c-7875-46bf-b77b-33698de0193d None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:36:09.084172 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414d0f8c-7875-46bf-b77b-33698de0193d None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:36:09.100795 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-b0d32295-c061-4abc-ace8-863e68993fb7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '77e7e8defac94cbd9457411dff1fd730', 'request_id': 'req-b0d32295-c061-4abc-ace8-863e68993fb7'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:36:09.165864 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258c59ff-1e49-4749-b3d4-9686acc0c096 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:36:09.167221 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258c59ff-1e49-4749-b3d4-9686acc0c096 admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:36:09.167502 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258c59ff-1e49-4749-b3d4-9686acc0c096 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:36:09.167786 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258c59ff-1e49-4749-b3d4-9686acc0c096 admin admin] PATH_INFO: `/v3/domains` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:36:09.169223 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258c59ff-1e49-4749-b3d4-9686acc0c096 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:36:09.169657 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258c59ff-1e49-4749-b3d4-9686acc0c096 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:36:09.172509 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258c59ff-1e49-4749-b3d4-9686acc0c096 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:36:09.185469 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 927/1835] 10.222.0.30 () {58 vars in 1153 bytes} [Tue Sep 6 04:36:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:36:09.190057 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258c59ff-1e49-4749-b3d4-9686acc0c096 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:36:09.199979 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258c59ff-1e49-4749-b3d4-9686acc0c096 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:36:09.204677 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe29624-862d-49e9-83d1-2e2e0c0ae773 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:36:09.205399 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe29624-862d-49e9-83d1-2e2e0c0ae773 admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:36:09.205752 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe29624-862d-49e9-83d1-2e2e0c0ae773 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:36:09.206099 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe29624-862d-49e9-83d1-2e2e0c0ae773 admin admin] PATH_INFO: `/v3/projects` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:36:09.206211 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-b0d32295-c061-4abc-ace8-863e68993fb7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '77e7e8defac94cbd9457411dff1fd730', 'request_id': 'req-b0d32295-c061-4abc-ace8-863e68993fb7'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:36:09.207029 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe29624-862d-49e9-83d1-2e2e0c0ae773 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:36:09.210410 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe29624-862d-49e9-83d1-2e2e0c0ae773 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:36:09.216601 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-b0d32295-c061-4abc-ace8-863e68993fb7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '77e7e8defac94cbd9457411dff1fd730', 'request_id': 'req-b0d32295-c061-4abc-ace8-863e68993fb7'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:36:09.246999 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-b0d32295-c061-4abc-ace8-863e68993fb7 admin admin] The token cache is being invalidate because project 77e7e8defac94cbd9457411dff1fd730 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81697) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 04:36:09.247374 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-b0d32295-c061-4abc-ace8-863e68993fb7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:36:09.248692 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-b0d32295-c061-4abc-ace8-863e68993fb7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:36:09.252340 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 909/1836] 10.222.0.30 () {58 vars in 1233 bytes} [Tue Sep 6 04:36:08 2022] DELETE /identity/v3/projects/77e7e8defac94cbd9457411dff1fd730 => generated 0 bytes in 486 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 04:36:09.258970 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d32295-c061-4abc-ace8-863e68993fb7 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:36:09.261711 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-b0d32295-c061-4abc-ace8-863e68993fb7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:36:09.280089 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d32295-c061-4abc-ace8-863e68993fb7 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:36:09.287138 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eec4a17-3960-48d2-9e3b-4d6bc2b1ddb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:36:09.287138 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eec4a17-3960-48d2-9e3b-4d6bc2b1ddb8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:36:09.287138 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eec4a17-3960-48d2-9e3b-4d6bc2b1ddb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:36:09.287138 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eec4a17-3960-48d2-9e3b-4d6bc2b1ddb8 admin admin] PATH_INFO: `/v3/users/0b25b2fd399f4778acc1727b802c7678` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:36:09.288004 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eec4a17-3960-48d2-9e3b-4d6bc2b1ddb8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0b25b2fd399f4778acc1727b802c7678)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:36:09.290041 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eec4a17-3960-48d2-9e3b-4d6bc2b1ddb8 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:36:09.341133 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 928/1837] 10.222.0.30 () {60 vars in 1153 bytes} [Tue Sep 6 04:36:09 2022] POST /identity/v3/projects => generated 385 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 04:36:09.345022 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe29624-862d-49e9-83d1-2e2e0c0ae773 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:36:09.346781 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-0fe29624-862d-49e9-83d1-2e2e0c0ae773 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:36:09.415918 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe29624-862d-49e9-83d1-2e2e0c0ae773 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:36:09.422435 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c68b7f-322a-4970-8ac8-3caffaf17cf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:36:09.424021 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c68b7f-322a-4970-8ac8-3caffaf17cf3 admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:36:09.424301 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c68b7f-322a-4970-8ac8-3caffaf17cf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:36:09.424571 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c68b7f-322a-4970-8ac8-3caffaf17cf3 admin admin] PATH_INFO: `/v3/users` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:36:09.425206 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c68b7f-322a-4970-8ac8-3caffaf17cf3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:36:09.427165 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c68b7f-322a-4970-8ac8-3caffaf17cf3 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:36:09.485929 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-5eec4a17-3960-48d2-9e3b-4d6bc2b1ddb8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0b25b2fd399f4778acc1727b802c7678'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:36:09.511275 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 929/1838] 10.222.0.30 () {60 vars in 1144 bytes} [Tue Sep 6 04:36:09 2022] POST /identity/v3/users => generated 293 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 04:36:09.512266 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c68b7f-322a-4970-8ac8-3caffaf17cf3 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:36:09.514007 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-b3c68b7f-322a-4970-8ac8-3caffaf17cf3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:36:09.527214 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c68b7f-322a-4970-8ac8-3caffaf17cf3 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:36:09.532196 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9586c1c-3670-40b0-9d54-956001d80a14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:36:09.532885 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9586c1c-3670-40b0-9d54-956001d80a14 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:36:09.533167 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9586c1c-3670-40b0-9d54-956001d80a14 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:36:09.533482 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9586c1c-3670-40b0-9d54-956001d80a14 admin admin] PATH_INFO: `/v3/projects/dd61b305100546eaa11b26e4a7e5054f` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:36:09.537182 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9586c1c-3670-40b0-9d54-956001d80a14 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dd61b305100546eaa11b26e4a7e5054f)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:36:09.537182 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9586c1c-3670-40b0-9d54-956001d80a14 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:36:09.611977 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-5eec4a17-3960-48d2-9e3b-4d6bc2b1ddb8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0b25b2fd399f4778acc1727b802c7678'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:36:09.612778 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-5eec4a17-3960-48d2-9e3b-4d6bc2b1ddb8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0b25b2fd399f4778acc1727b802c7678'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:36:09.621394 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-5eec4a17-3960-48d2-9e3b-4d6bc2b1ddb8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0b25b2fd399f4778acc1727b802c7678'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:36:09.634698 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-5eec4a17-3960-48d2-9e3b-4d6bc2b1ddb8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0b25b2fd399f4778acc1727b802c7678'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:36:09.642613 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 910/1839] 10.222.0.30 () {58 vars in 1224 bytes} [Tue Sep 6 04:36:09 2022] DELETE /identity/v3/users/0b25b2fd399f4778acc1727b802c7678 => generated 0 bytes in 385 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 04:36:09.648857 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eec4a17-3960-48d2-9e3b-4d6bc2b1ddb8 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:36:09.648857 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-5eec4a17-3960-48d2-9e3b-4d6bc2b1ddb8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:36:09.662595 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-c9586c1c-3670-40b0-9d54-956001d80a14 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dd61b305100546eaa11b26e4a7e5054f', 'request_id': 'req-c9586c1c-3670-40b0-9d54-956001d80a14'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:36:09.669604 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-c9586c1c-3670-40b0-9d54-956001d80a14 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dd61b305100546eaa11b26e4a7e5054f', 'request_id': 'req-c9586c1c-3670-40b0-9d54-956001d80a14'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:36:09.705024 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:36:09.718258 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eec4a17-3960-48d2-9e3b-4d6bc2b1ddb8 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:36:09.723423 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab4a9fb-4e20-42ae-8d5e-95d451287364 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:36:09.724046 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab4a9fb-4e20-42ae-8d5e-95d451287364 admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:36:09.724324 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab4a9fb-4e20-42ae-8d5e-95d451287364 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:36:09.725432 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab4a9fb-4e20-42ae-8d5e-95d451287364 admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:36:09.726091 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab4a9fb-4e20-42ae-8d5e-95d451287364 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:36:09.727733 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab4a9fb-4e20-42ae-8d5e-95d451287364 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:36:09.738512 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 911/1840] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Sep 6 04:36:09 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 04:36:09.742517 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab4a9fb-4e20-42ae-8d5e-95d451287364 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:36:09.752802 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab4a9fb-4e20-42ae-8d5e-95d451287364 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:36:09.761469 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e52952f-dc5f-438c-af0a-99200ab62eec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:36:09.762180 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e52952f-dc5f-438c-af0a-99200ab62eec admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:36:09.762720 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e52952f-dc5f-438c-af0a-99200ab62eec admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:36:09.762851 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e52952f-dc5f-438c-af0a-99200ab62eec admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:36:09.763896 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e52952f-dc5f-438c-af0a-99200ab62eec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:36:09.766205 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e52952f-dc5f-438c-af0a-99200ab62eec admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:36:09.777694 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 912/1841] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Sep 6 04:36:09 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 06 04:36:09.781980 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e52952f-dc5f-438c-af0a-99200ab62eec admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:36:09.791693 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e52952f-dc5f-438c-af0a-99200ab62eec admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:36:09.796382 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9591252-4b83-484f-bb59-4fbb77e45c4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:36:09.797030 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9591252-4b83-484f-bb59-4fbb77e45c4a admin admin] REQUEST_METHOD: `PUT` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:36:09.797326 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9591252-4b83-484f-bb59-4fbb77e45c4a admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:36:09.797611 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9591252-4b83-484f-bb59-4fbb77e45c4a admin admin] PATH_INFO: `/v3/projects/8118794c3b26418c92de08614b25d685/users/f1de2e8b04e8434ba1e47322c05832b9/roles/29607b2de39f40d68bb6ceef64b8a732` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:36:09.812374 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9591252-4b83-484f-bb59-4fbb77e45c4a admin admin] RBAC: Authorizing `identity:create_grant(project_id=8118794c3b26418c92de08614b25d685, user_id=f1de2e8b04e8434ba1e47322c05832b9, role_id=29607b2de39f40d68bb6ceef64b8a732)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:36:09.816241 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9591252-4b83-484f-bb59-4fbb77e45c4a admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:36:09.830638 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-c9586c1c-3670-40b0-9d54-956001d80a14 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dd61b305100546eaa11b26e4a7e5054f', 'request_id': 'req-c9586c1c-3670-40b0-9d54-956001d80a14'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:36:09.846033 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-c9586c1c-3670-40b0-9d54-956001d80a14 admin admin] The token cache is being invalidate because project dd61b305100546eaa11b26e4a7e5054f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81698) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 04:36:09.846575 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-c9586c1c-3670-40b0-9d54-956001d80a14 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:36:09.847138 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-c9586c1c-3670-40b0-9d54-956001d80a14 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:36:09.848750 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 930/1842] 10.222.0.30 () {58 vars in 1233 bytes} [Tue Sep 6 04:36:09 2022] DELETE /identity/v3/projects/dd61b305100546eaa11b26e4a7e5054f => generated 0 bytes in 337 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 04:36:09.853482 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c75b42-09f8-4d17-9143-e6d70ec036b8 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:36:09.857029 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c75b42-09f8-4d17-9143-e6d70ec036b8 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:36:09.857029 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c75b42-09f8-4d17-9143-e6d70ec036b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:36:09.878968 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:36:09.879929 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-63c75b42-09f8-4d17-9143-e6d70ec036b8 None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:36:09.881270 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-63c75b42-09f8-4d17-9143-e6d70ec036b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:36:09.892573 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 931/1843] 10.222.0.30 () {58 vars in 958 bytes} [Tue Sep 6 04:36:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 04:36:09.902129 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4b88ba-ad65-4112-b686-f93856b0e004 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:36:09.902333 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4b88ba-ad65-4112-b686-f93856b0e004 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:36:09.903658 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4b88ba-ad65-4112-b686-f93856b0e004 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:36:09.912345 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 913/1844] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Sep 6 04:36:09 2022] PUT /identity/v3/projects/8118794c3b26418c92de08614b25d685/users/f1de2e8b04e8434ba1e47322c05832b9/roles/29607b2de39f40d68bb6ceef64b8a732 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 04:36:09.923060 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5874c55-0fb9-402c-b137-ec2a49aeb8d2 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:36:09.925262 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5874c55-0fb9-402c-b137-ec2a49aeb8d2 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:36:09.925262 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5874c55-0fb9-402c-b137-ec2a49aeb8d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:36:09.925491 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-fb4b88ba-ad65-4112-b686-f93856b0e004 None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:36:09.926739 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-fb4b88ba-ad65-4112-b686-f93856b0e004 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:36:09.956034 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-b5874c55-0fb9-402c-b137-ec2a49aeb8d2 None None] MFA Rules not processed for user `f1de2e8b04e8434ba1e47322c05832b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:36:09.957248 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-b5874c55-0fb9-402c-b137-ec2a49aeb8d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:36:10.027647 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 932/1845] 10.222.0.30 () {58 vars in 958 bytes} [Tue Sep 6 04:36:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 04:36:10.031852 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 914/1846] 10.222.0.30 () {58 vars in 958 bytes} [Tue Sep 6 04:36:09 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 04:36:10.031989 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4b88ba-ad65-4112-b686-f93856b0e004 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:36:10.042250 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5874c55-0fb9-402c-b137-ec2a49aeb8d2 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:36:10.043232 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4b88ba-ad65-4112-b686-f93856b0e004 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:36:10.043844 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-b5874c55-0fb9-402c-b137-ec2a49aeb8d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:36:10.048493 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2942fc84-74dc-4f2b-80c1-659c65faaa99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:36:10.049406 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2942fc84-74dc-4f2b-80c1-659c65faaa99 admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:36:10.049695 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2942fc84-74dc-4f2b-80c1-659c65faaa99 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:36:10.049974 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2942fc84-74dc-4f2b-80c1-659c65faaa99 admin admin] PATH_INFO: `/v3/domains` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:36:10.050633 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2942fc84-74dc-4f2b-80c1-659c65faaa99 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:36:10.050934 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2942fc84-74dc-4f2b-80c1-659c65faaa99 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:36:10.052545 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2942fc84-74dc-4f2b-80c1-659c65faaa99 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:36:10.076403 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5874c55-0fb9-402c-b137-ec2a49aeb8d2 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:36:10.086852 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 933/1847] 10.222.0.30 () {58 vars in 1153 bytes} [Tue Sep 6 04:36:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:36:10.089709 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2942fc84-74dc-4f2b-80c1-659c65faaa99 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:36:10.091411 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89dde7a-e702-4155-b37a-d984c24585da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:36:10.092311 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89dde7a-e702-4155-b37a-d984c24585da admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:36:10.092596 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89dde7a-e702-4155-b37a-d984c24585da admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:36:10.092869 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89dde7a-e702-4155-b37a-d984c24585da admin admin] PATH_INFO: `/v3/projects` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:36:10.093559 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89dde7a-e702-4155-b37a-d984c24585da admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:36:10.095469 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89dde7a-e702-4155-b37a-d984c24585da admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:36:10.114214 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2942fc84-74dc-4f2b-80c1-659c65faaa99 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:36:10.122236 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c9abec-9507-4a91-b3a9-84b225d60b6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:36:10.123458 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c9abec-9507-4a91-b3a9-84b225d60b6e admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:36:10.123458 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c9abec-9507-4a91-b3a9-84b225d60b6e admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:36:10.123745 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c9abec-9507-4a91-b3a9-84b225d60b6e admin admin] PATH_INFO: `/v3/projects` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:36:10.124187 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c9abec-9507-4a91-b3a9-84b225d60b6e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:36:10.127039 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c9abec-9507-4a91-b3a9-84b225d60b6e admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:36:10.224455 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 915/1848] 10.222.0.30 () {60 vars in 1153 bytes} [Tue Sep 6 04:36:10 2022] POST /identity/v3/projects => generated 385 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 04:36:10.229160 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 934/1849] 10.222.0.30 () {60 vars in 1153 bytes} [Tue Sep 6 04:36:10 2022] POST /identity/v3/projects => generated 409 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 04:36:10.230481 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c9abec-9507-4a91-b3a9-84b225d60b6e admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:36:10.232968 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89dde7a-e702-4155-b37a-d984c24585da admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:36:10.331797 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89dde7a-e702-4155-b37a-d984c24585da admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:36:10.335829 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c9abec-9507-4a91-b3a9-84b225d60b6e admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:36:10.336683 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab795a1-4cec-478d-b57d-1b1513c08c7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:36:10.337370 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab795a1-4cec-478d-b57d-1b1513c08c7c admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:36:10.337899 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab795a1-4cec-478d-b57d-1b1513c08c7c admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:36:10.338327 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab795a1-4cec-478d-b57d-1b1513c08c7c admin admin] PATH_INFO: `/v3/users` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:36:10.339093 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab795a1-4cec-478d-b57d-1b1513c08c7c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:36:10.341432 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab795a1-4cec-478d-b57d-1b1513c08c7c admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:36:10.342875 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d361ebb-f29b-4335-b871-b21b7f7f56f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:36:10.344227 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d361ebb-f29b-4335-b871-b21b7f7f56f5 admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:36:10.345016 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d361ebb-f29b-4335-b871-b21b7f7f56f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:36:10.345694 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d361ebb-f29b-4335-b871-b21b7f7f56f5 admin admin] PATH_INFO: `/v3/users` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:36:10.347090 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d361ebb-f29b-4335-b871-b21b7f7f56f5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:36:10.349222 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d361ebb-f29b-4335-b871-b21b7f7f56f5 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:36:10.479704 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 935/1850] 10.222.0.30 () {60 vars in 1144 bytes} [Tue Sep 6 04:36:10 2022] POST /identity/v3/users => generated 299 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 04:36:10.480395 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 916/1851] 10.222.0.30 () {60 vars in 1144 bytes} [Tue Sep 6 04:36:10 2022] POST /identity/v3/users => generated 305 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 04:36:10.480738 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d361ebb-f29b-4335-b871-b21b7f7f56f5 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:36:10.486063 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab795a1-4cec-478d-b57d-1b1513c08c7c admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:36:10.488801 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-2d361ebb-f29b-4335-b871-b21b7f7f56f5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:36:10.508882 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab795a1-4cec-478d-b57d-1b1513c08c7c admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:36:10.515066 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d361ebb-f29b-4335-b871-b21b7f7f56f5 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:36:10.515944 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae52d19-0181-4bde-8ee8-911bd4bf5e66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:36:10.516806 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae52d19-0181-4bde-8ee8-911bd4bf5e66 admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:36:10.517354 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae52d19-0181-4bde-8ee8-911bd4bf5e66 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:36:10.517857 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae52d19-0181-4bde-8ee8-911bd4bf5e66 admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:36:10.518714 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae52d19-0181-4bde-8ee8-911bd4bf5e66 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:36:10.520691 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae52d19-0181-4bde-8ee8-911bd4bf5e66 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:36:10.532041 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab55b566-6867-4505-98d6-7f37c39e65bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:36:10.533107 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab55b566-6867-4505-98d6-7f37c39e65bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:36:10.533654 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab55b566-6867-4505-98d6-7f37c39e65bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:36:10.539075 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab55b566-6867-4505-98d6-7f37c39e65bd admin admin] PATH_INFO: `/v3/projects/850d080e1c204836bbaec7e8b50a7f1e` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:36:10.539075 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab55b566-6867-4505-98d6-7f37c39e65bd admin admin] RBAC: Authorizing `identity:delete_project(project_id=850d080e1c204836bbaec7e8b50a7f1e)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:36:10.540220 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab55b566-6867-4505-98d6-7f37c39e65bd admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:36:10.568639 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 917/1852] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Sep 6 04:36:10 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 04:36:10.568639 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae52d19-0181-4bde-8ee8-911bd4bf5e66 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:36:10.592044 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae52d19-0181-4bde-8ee8-911bd4bf5e66 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:36:10.607059 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb9bf90-e1f0-48a4-8a5c-030d5e08843c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:36:10.609414 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb9bf90-e1f0-48a4-8a5c-030d5e08843c admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:36:10.609705 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb9bf90-e1f0-48a4-8a5c-030d5e08843c admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:36:10.609984 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb9bf90-e1f0-48a4-8a5c-030d5e08843c admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:36:10.610646 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb9bf90-e1f0-48a4-8a5c-030d5e08843c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:36:10.613237 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb9bf90-e1f0-48a4-8a5c-030d5e08843c admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:36:10.654670 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 918/1853] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Sep 6 04:36:10 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 04:36:10.654670 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb9bf90-e1f0-48a4-8a5c-030d5e08843c admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:36:10.667703 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb9bf90-e1f0-48a4-8a5c-030d5e08843c admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:36:10.673784 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ee82f8-1e83-484e-9982-6b1479ce46a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:36:10.674629 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ee82f8-1e83-484e-9982-6b1479ce46a9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:36:10.675310 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ee82f8-1e83-484e-9982-6b1479ce46a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:36:10.675860 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ee82f8-1e83-484e-9982-6b1479ce46a9 admin admin] PATH_INFO: `/v3/projects/9d9dd6c3c81e4506a5c198e670c461fa/users/daeab80654c3421687cdcaeef38bc63e/roles/ea4c814713b149108e488de855b025e4` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:36:10.700226 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ee82f8-1e83-484e-9982-6b1479ce46a9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9d9dd6c3c81e4506a5c198e670c461fa, user_id=daeab80654c3421687cdcaeef38bc63e, role_id=ea4c814713b149108e488de855b025e4)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:36:10.704275 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ee82f8-1e83-484e-9982-6b1479ce46a9 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:36:10.717882 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-ab55b566-6867-4505-98d6-7f37c39e65bd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '850d080e1c204836bbaec7e8b50a7f1e', 'request_id': 'req-ab55b566-6867-4505-98d6-7f37c39e65bd'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:36:10.724738 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-ab55b566-6867-4505-98d6-7f37c39e65bd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '850d080e1c204836bbaec7e8b50a7f1e', 'request_id': 'req-ab55b566-6867-4505-98d6-7f37c39e65bd'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:36:10.794572 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 919/1854] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Sep 6 04:36:10 2022] PUT /identity/v3/projects/9d9dd6c3c81e4506a5c198e670c461fa/users/daeab80654c3421687cdcaeef38bc63e/roles/ea4c814713b149108e488de855b025e4 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 04:36:10.795870 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ee82f8-1e83-484e-9982-6b1479ce46a9 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:36:10.879367 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ee82f8-1e83-484e-9982-6b1479ce46a9 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:36:10.887643 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-ab55b566-6867-4505-98d6-7f37c39e65bd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '850d080e1c204836bbaec7e8b50a7f1e', 'request_id': 'req-ab55b566-6867-4505-98d6-7f37c39e65bd'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:36:10.892786 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b1ade4-7732-4460-8dd9-10fb8d8de87c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:36:10.893446 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b1ade4-7732-4460-8dd9-10fb8d8de87c admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:36:10.893730 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b1ade4-7732-4460-8dd9-10fb8d8de87c admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:36:10.894007 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b1ade4-7732-4460-8dd9-10fb8d8de87c admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:36:10.896608 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b1ade4-7732-4460-8dd9-10fb8d8de87c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:36:10.898275 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b1ade4-7732-4460-8dd9-10fb8d8de87c admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:36:10.901929 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-ab55b566-6867-4505-98d6-7f37c39e65bd admin admin] The token cache is being invalidate because project 850d080e1c204836bbaec7e8b50a7f1e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81698) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 04:36:10.902377 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-ab55b566-6867-4505-98d6-7f37c39e65bd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:36:10.903046 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-ab55b566-6867-4505-98d6-7f37c39e65bd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:36:10.904773 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 936/1855] 10.222.0.30 () {58 vars in 1233 bytes} [Tue Sep 6 04:36:10 2022] DELETE /identity/v3/projects/850d080e1c204836bbaec7e8b50a7f1e => generated 0 bytes in 424 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 04:36:10.905412 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab55b566-6867-4505-98d6-7f37c39e65bd admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:36:10.906748 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-ab55b566-6867-4505-98d6-7f37c39e65bd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:36:10.912629 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 920/1856] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Sep 6 04:36:10 2022] GET /identity/v3/roles => generated 1420 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 04:36:10.916425 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b1ade4-7732-4460-8dd9-10fb8d8de87c admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:36:10.917964 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-11b1ade4-7732-4460-8dd9-10fb8d8de87c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:36:10.936608 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b1ade4-7732-4460-8dd9-10fb8d8de87c admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:36:10.941572 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4d41ad-3c25-4aac-b664-751eed5c0247 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:36:10.945483 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4d41ad-3c25-4aac-b664-751eed5c0247 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:36:10.945483 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4d41ad-3c25-4aac-b664-751eed5c0247 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:36:10.945483 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4d41ad-3c25-4aac-b664-751eed5c0247 admin admin] PATH_INFO: `/v3/users/1a57fbc71cac44449c48e783ad17054c` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:36:10.945483 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4d41ad-3c25-4aac-b664-751eed5c0247 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1a57fbc71cac44449c48e783ad17054c)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:36:10.946416 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4d41ad-3c25-4aac-b664-751eed5c0247 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:36:10.990176 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab55b566-6867-4505-98d6-7f37c39e65bd admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:36:11.002520 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21110fb-29af-41f8-b44c-1622353004d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:36:11.003068 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21110fb-29af-41f8-b44c-1622353004d0 admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:36:11.003366 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21110fb-29af-41f8-b44c-1622353004d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:36:11.003655 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21110fb-29af-41f8-b44c-1622353004d0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:36:11.004308 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21110fb-29af-41f8-b44c-1622353004d0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:36:11.005952 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21110fb-29af-41f8-b44c-1622353004d0 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:36:11.016870 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 937/1857] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Sep 6 04:36:10 2022] GET /identity/v3/roles => generated 1420 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 04:36:11.018231 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21110fb-29af-41f8-b44c-1622353004d0 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:36:11.020029 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-a21110fb-29af-41f8-b44c-1622353004d0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:36:11.032770 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21110fb-29af-41f8-b44c-1622353004d0 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:36:11.037666 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb0f5b1-e3da-4aa6-94a1-34083bb3e06c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:36:11.038322 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb0f5b1-e3da-4aa6-94a1-34083bb3e06c admin admin] REQUEST_METHOD: `PUT` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:36:11.038600 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb0f5b1-e3da-4aa6-94a1-34083bb3e06c admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:36:11.038870 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb0f5b1-e3da-4aa6-94a1-34083bb3e06c admin admin] PATH_INFO: `/v3/projects/52bb42a8f0ec457483a21c769a9f4561/users/7ebc0166ad01461294d58ad5c5998633/roles/29607b2de39f40d68bb6ceef64b8a732` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:36:11.055984 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb0f5b1-e3da-4aa6-94a1-34083bb3e06c admin admin] RBAC: Authorizing `identity:create_grant(project_id=52bb42a8f0ec457483a21c769a9f4561, user_id=7ebc0166ad01461294d58ad5c5998633, role_id=29607b2de39f40d68bb6ceef64b8a732)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:36:11.060406 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb0f5b1-e3da-4aa6-94a1-34083bb3e06c admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:36:11.081818 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:36:11.156413 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-8e4d41ad-3c25-4aac-b664-751eed5c0247 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1a57fbc71cac44449c48e783ad17054c'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:36:11.184695 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 938/1858] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Sep 6 04:36:11 2022] PUT /identity/v3/projects/52bb42a8f0ec457483a21c769a9f4561/users/7ebc0166ad01461294d58ad5c5998633/roles/29607b2de39f40d68bb6ceef64b8a732 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 04:36:11.185984 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb0f5b1-e3da-4aa6-94a1-34083bb3e06c admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:36:11.259070 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:36:11.262568 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-8e4d41ad-3c25-4aac-b664-751eed5c0247 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1a57fbc71cac44449c48e783ad17054c'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:36:11.262568 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-8e4d41ad-3c25-4aac-b664-751eed5c0247 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1a57fbc71cac44449c48e783ad17054c'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:36:11.262773 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb0f5b1-e3da-4aa6-94a1-34083bb3e06c admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:36:11.268438 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d05788-79f5-4513-886d-b701823b9812 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:36:11.269173 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d05788-79f5-4513-886d-b701823b9812 admin admin] REQUEST_METHOD: `PUT` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:36:11.269481 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d05788-79f5-4513-886d-b701823b9812 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:36:11.269593 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-8e4d41ad-3c25-4aac-b664-751eed5c0247 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1a57fbc71cac44449c48e783ad17054c'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:36:11.269755 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d05788-79f5-4513-886d-b701823b9812 admin admin] PATH_INFO: `/v3/projects/9d9dd6c3c81e4506a5c198e670c461fa/users/daeab80654c3421687cdcaeef38bc63e/roles/ea4c814713b149108e488de855b025e4` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:36:11.273194 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d05788-79f5-4513-886d-b701823b9812 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9d9dd6c3c81e4506a5c198e670c461fa, user_id=daeab80654c3421687cdcaeef38bc63e, role_id=ea4c814713b149108e488de855b025e4)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:36:11.275005 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d05788-79f5-4513-886d-b701823b9812 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:36:11.285849 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-8e4d41ad-3c25-4aac-b664-751eed5c0247 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1a57fbc71cac44449c48e783ad17054c'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:36:11.287359 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 939/1859] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Sep 6 04:36:11 2022] PUT /identity/v3/projects/9d9dd6c3c81e4506a5c198e670c461fa/users/daeab80654c3421687cdcaeef38bc63e/roles/ea4c814713b149108e488de855b025e4 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 04:36:11.288708 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a3dd91-18d9-44c9-914e-3dae1890f273 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:36:11.288987 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a3dd91-18d9-44c9-914e-3dae1890f273 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:36:11.289259 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a3dd91-18d9-44c9-914e-3dae1890f273 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:36:11.303864 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 921/1860] 10.222.0.30 () {58 vars in 1224 bytes} [Tue Sep 6 04:36:10 2022] DELETE /identity/v3/users/1a57fbc71cac44449c48e783ad17054c => generated 0 bytes in 391 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 04:36:11.305576 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31694e7d-96d2-4f34-aa14-898e34ec1a9d None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:36:11.305968 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31694e7d-96d2-4f34-aa14-898e34ec1a9d None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:36:11.306344 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31694e7d-96d2-4f34-aa14-898e34ec1a9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:36:11.339339 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-41a3dd91-18d9-44c9-914e-3dae1890f273 None None] MFA Rules not processed for user `7ebc0166ad01461294d58ad5c5998633`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:36:11.340477 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-41a3dd91-18d9-44c9-914e-3dae1890f273 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:36:11.367038 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-31694e7d-96d2-4f34-aa14-898e34ec1a9d None None] MFA Rules not processed for user `daeab80654c3421687cdcaeef38bc63e`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:36:11.371334 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-31694e7d-96d2-4f34-aa14-898e34ec1a9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:36:11.440460 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 940/1861] 10.222.0.30 () {58 vars in 958 bytes} [Tue Sep 6 04:36:11 2022] POST /identity/v3/auth/tokens => generated 3286 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 04:36:11.445057 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a3dd91-18d9-44c9-914e-3dae1890f273 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:36:11.446617 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-41a3dd91-18d9-44c9-914e-3dae1890f273 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:36:11.484130 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 922/1862] 10.222.0.30 () {58 vars in 958 bytes} [Tue Sep 6 04:36:11 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 04:36:11.500634 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31694e7d-96d2-4f34-aa14-898e34ec1a9d None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:36:11.502658 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-31694e7d-96d2-4f34-aa14-898e34ec1a9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:36:11.575327 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a3dd91-18d9-44c9-914e-3dae1890f273 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:36:11.580455 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ef8823-482c-42d7-8e53-2d989408d82f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:36:11.581076 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ef8823-482c-42d7-8e53-2d989408d82f admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:36:11.581568 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ef8823-482c-42d7-8e53-2d989408d82f admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:36:11.581989 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ef8823-482c-42d7-8e53-2d989408d82f admin admin] PATH_INFO: `/v3/projects` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:36:11.582634 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ef8823-482c-42d7-8e53-2d989408d82f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:36:11.584540 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ef8823-482c-42d7-8e53-2d989408d82f admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:36:11.590228 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31694e7d-96d2-4f34-aa14-898e34ec1a9d None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:36:11.594367 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28608859-8a17-430d-9008-ebae40a266c9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd09b5ab675634a2eb4b03755743f7d0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b707733ce65c42edbc1fd95ad0a1c06c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:36:11.595073 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28608859-8a17-430d-9008-ebae40a266c9 service cinder] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:36:11.595462 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28608859-8a17-430d-9008-ebae40a266c9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:36:11.595824 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28608859-8a17-430d-9008-ebae40a266c9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:36:11.609181 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28608859-8a17-430d-9008-ebae40a266c9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:36:11.610653 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28608859-8a17-430d-9008-ebae40a266c9 service cinder] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:36:11.622673 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 923/1863] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Sep 6 04:36:11 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 06 04:36:11.696591 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28608859-8a17-430d-9008-ebae40a266c9 service cinder] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:36:11.703891 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 941/1864] 10.222.0.30 () {60 vars in 1153 bytes} [Tue Sep 6 04:36:11 2022] POST /identity/v3/projects => generated 407 bytes in 259 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 04:36:11.710448 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ef8823-482c-42d7-8e53-2d989408d82f admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:36:11.712520 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-76ef8823-482c-42d7-8e53-2d989408d82f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:36:11.715076 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28608859-8a17-430d-9008-ebae40a266c9 service cinder] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:36:11.785838 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f3b5ad-0e4a-42cb-85b6-827af134cbab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:36:11.786759 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f3b5ad-0e4a-42cb-85b6-827af134cbab admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:36:11.787048 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f3b5ad-0e4a-42cb-85b6-827af134cbab admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:36:11.787324 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f3b5ad-0e4a-42cb-85b6-827af134cbab admin admin] PATH_INFO: `/v3/users` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:36:11.788000 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f3b5ad-0e4a-42cb-85b6-827af134cbab admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:36:11.789996 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f3b5ad-0e4a-42cb-85b6-827af134cbab admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:36:11.792146 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ef8823-482c-42d7-8e53-2d989408d82f admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:36:11.800000 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbf03bb-07cd-403d-8f00-94dc3bced822 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:36:11.801724 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbf03bb-07cd-403d-8f00-94dc3bced822 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:36:11.802144 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbf03bb-07cd-403d-8f00-94dc3bced822 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:36:11.802433 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbf03bb-07cd-403d-8f00-94dc3bced822 admin admin] PATH_INFO: `/v3/users/f1de2e8b04e8434ba1e47322c05832b9` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:36:11.803877 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cbf03bb-07cd-403d-8f00-94dc3bced822 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f1de2e8b04e8434ba1e47322c05832b9)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:36:11.806817 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cbf03bb-07cd-403d-8f00-94dc3bced822 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:36:11.873700 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 924/1865] 10.222.0.30 () {60 vars in 1144 bytes} [Tue Sep 6 04:36:11 2022] POST /identity/v3/users => generated 310 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 04:36:11.874993 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f3b5ad-0e4a-42cb-85b6-827af134cbab admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:36:11.885784 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f3b5ad-0e4a-42cb-85b6-827af134cbab admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:36:11.890430 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8cc96c-d915-47cb-84eb-4b01d53bc8c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:36:11.891039 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8cc96c-d915-47cb-84eb-4b01d53bc8c0 admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:36:11.891319 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8cc96c-d915-47cb-84eb-4b01d53bc8c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:36:11.891594 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8cc96c-d915-47cb-84eb-4b01d53bc8c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:36:11.892803 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8cc96c-d915-47cb-84eb-4b01d53bc8c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:36:11.894441 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8cc96c-d915-47cb-84eb-4b01d53bc8c0 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:36:11.905183 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 925/1866] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Sep 6 04:36:11 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 06 04:36:11.907175 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8cc96c-d915-47cb-84eb-4b01d53bc8c0 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:36:11.909049 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-9a8cc96c-d915-47cb-84eb-4b01d53bc8c0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:36:11.923593 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8cc96c-d915-47cb-84eb-4b01d53bc8c0 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:36:11.928693 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f9e641-bc8b-4e88-94c2-2ee4cc9bc33e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:36:11.929362 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f9e641-bc8b-4e88-94c2-2ee4cc9bc33e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:36:11.932625 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f9e641-bc8b-4e88-94c2-2ee4cc9bc33e admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:36:11.932625 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f9e641-bc8b-4e88-94c2-2ee4cc9bc33e admin admin] PATH_INFO: `/v3/projects/fffff36fe18b457a910f7447db81ccc5` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:36:11.932625 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f9e641-bc8b-4e88-94c2-2ee4cc9bc33e admin admin] RBAC: Authorizing `identity:delete_project(project_id=fffff36fe18b457a910f7447db81ccc5)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:36:11.933758 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f9e641-bc8b-4e88-94c2-2ee4cc9bc33e admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:36:11.999163 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-5cbf03bb-07cd-403d-8f00-94dc3bced822 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f1de2e8b04e8434ba1e47322c05832b9'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:36:12.058753 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-d8f9e641-bc8b-4e88-94c2-2ee4cc9bc33e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fffff36fe18b457a910f7447db81ccc5', 'request_id': 'req-d8f9e641-bc8b-4e88-94c2-2ee4cc9bc33e'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:36:12.071191 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-5cbf03bb-07cd-403d-8f00-94dc3bced822 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f1de2e8b04e8434ba1e47322c05832b9'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:36:12.071841 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-5cbf03bb-07cd-403d-8f00-94dc3bced822 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f1de2e8b04e8434ba1e47322c05832b9'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:36:12.086398 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-5cbf03bb-07cd-403d-8f00-94dc3bced822 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f1de2e8b04e8434ba1e47322c05832b9'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:36:12.094850 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-5cbf03bb-07cd-403d-8f00-94dc3bced822 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f1de2e8b04e8434ba1e47322c05832b9'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:36:12.101119 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 942/1867] 10.222.0.30 () {58 vars in 1224 bytes} [Tue Sep 6 04:36:11 2022] DELETE /identity/v3/users/f1de2e8b04e8434ba1e47322c05832b9 => generated 0 bytes in 391 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 04:36:12.101678 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbf03bb-07cd-403d-8f00-94dc3bced822 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:36:12.103016 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-5cbf03bb-07cd-403d-8f00-94dc3bced822 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:36:12.173760 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbf03bb-07cd-403d-8f00-94dc3bced822 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:36:12.179161 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3b320a-d968-485f-a354-d1cb8a79b30c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:36:12.180049 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3b320a-d968-485f-a354-d1cb8a79b30c admin admin] REQUEST_METHOD: `PUT` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:36:12.180501 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3b320a-d968-485f-a354-d1cb8a79b30c admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:36:12.180906 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3b320a-d968-485f-a354-d1cb8a79b30c admin admin] PATH_INFO: `/v3/projects/f4d8325f73d44d6db02cc9e937e3f218/users/088224d4ba6c4dc599f1c184d534f089/roles/ea4c814713b149108e488de855b025e4` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:36:12.196375 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-d8f9e641-bc8b-4e88-94c2-2ee4cc9bc33e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fffff36fe18b457a910f7447db81ccc5', 'request_id': 'req-d8f9e641-bc8b-4e88-94c2-2ee4cc9bc33e'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:36:12.200841 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3b320a-d968-485f-a354-d1cb8a79b30c admin admin] RBAC: Authorizing `identity:create_grant(project_id=f4d8325f73d44d6db02cc9e937e3f218, user_id=088224d4ba6c4dc599f1c184d534f089, role_id=ea4c814713b149108e488de855b025e4)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:36:12.207842 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-d8f9e641-bc8b-4e88-94c2-2ee4cc9bc33e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fffff36fe18b457a910f7447db81ccc5', 'request_id': 'req-d8f9e641-bc8b-4e88-94c2-2ee4cc9bc33e'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:36:12.208087 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3b320a-d968-485f-a354-d1cb8a79b30c admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:36:12.229624 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-d8f9e641-bc8b-4e88-94c2-2ee4cc9bc33e admin admin] The token cache is being invalidate because project fffff36fe18b457a910f7447db81ccc5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81697) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 04:36:12.229624 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-d8f9e641-bc8b-4e88-94c2-2ee4cc9bc33e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:36:12.229624 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-d8f9e641-bc8b-4e88-94c2-2ee4cc9bc33e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:36:12.229624 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 926/1868] 10.222.0.30 () {58 vars in 1233 bytes} [Tue Sep 6 04:36:11 2022] DELETE /identity/v3/projects/fffff36fe18b457a910f7447db81ccc5 => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 04:36:12.229624 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f4f978-1db8-4341-8584-e03e3185c463 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:36:12.230012 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f4f978-1db8-4341-8584-e03e3185c463 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:36:12.230012 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f4f978-1db8-4341-8584-e03e3185c463 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:36:12.247990 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-e4f4f978-1db8-4341-8584-e03e3185c463 None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:36:12.249362 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-e4f4f978-1db8-4341-8584-e03e3185c463 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:36:12.257668 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 927/1869] 10.222.0.30 () {58 vars in 958 bytes} [Tue Sep 6 04:36:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 04:36:12.259124 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c3421b-d046-40c3-8a09-231f9c4d5bcc None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:36:12.259412 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c3421b-d046-40c3-8a09-231f9c4d5bcc None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:36:12.259682 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c3421b-d046-40c3-8a09-231f9c4d5bcc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:36:12.287189 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-31c3421b-d046-40c3-8a09-231f9c4d5bcc None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:36:12.288398 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-31c3421b-d046-40c3-8a09-231f9c4d5bcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:36:12.297122 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 928/1870] 10.222.0.30 () {58 vars in 958 bytes} [Tue Sep 6 04:36:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 04:36:12.297984 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c3421b-d046-40c3-8a09-231f9c4d5bcc None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:36:12.306013 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-31c3421b-d046-40c3-8a09-231f9c4d5bcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:36:12.351951 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 943/1871] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Sep 6 04:36:12 2022] PUT /identity/v3/projects/f4d8325f73d44d6db02cc9e937e3f218/users/088224d4ba6c4dc599f1c184d534f089/roles/ea4c814713b149108e488de855b025e4 => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 04:36:12.353481 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77c37ca-ad33-4214-a5b2-bf98bb4ef8eb None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:36:12.354351 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77c37ca-ad33-4214-a5b2-bf98bb4ef8eb None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:36:12.354656 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77c37ca-ad33-4214-a5b2-bf98bb4ef8eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:36:12.382852 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-b77c37ca-ad33-4214-a5b2-bf98bb4ef8eb None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:36:12.385899 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-b77c37ca-ad33-4214-a5b2-bf98bb4ef8eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:36:12.386060 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c3421b-d046-40c3-8a09-231f9c4d5bcc None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:36:12.453252 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf27032-0542-41c0-9e90-2fed38c14a1c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90492440f6a6471e8392361c098428aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b707733ce65c42edbc1fd95ad0a1c06c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:36:12.453891 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf27032-0542-41c0-9e90-2fed38c14a1c service neutron] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:36:12.454176 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf27032-0542-41c0-9e90-2fed38c14a1c service neutron] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:36:12.454449 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf27032-0542-41c0-9e90-2fed38c14a1c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:36:12.516691 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 944/1872] 10.222.0.30 () {58 vars in 958 bytes} [Tue Sep 6 04:36:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 04:36:12.518205 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77c37ca-ad33-4214-a5b2-bf98bb4ef8eb None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:36:12.520139 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-b77c37ca-ad33-4214-a5b2-bf98bb4ef8eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:36:12.530966 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf27032-0542-41c0-9e90-2fed38c14a1c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:36:12.532400 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf27032-0542-41c0-9e90-2fed38c14a1c service neutron] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:36:12.532797 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77c37ca-ad33-4214-a5b2-bf98bb4ef8eb None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:36:12.541761 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6774952-9832-4ca6-95d2-ad31034bee07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:36:12.542463 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6774952-9832-4ca6-95d2-ad31034bee07 admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:36:12.543057 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6774952-9832-4ca6-95d2-ad31034bee07 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:36:12.543405 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6774952-9832-4ca6-95d2-ad31034bee07 admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:36:12.544123 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6774952-9832-4ca6-95d2-ad31034bee07 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:36:12.545814 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6774952-9832-4ca6-95d2-ad31034bee07 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:36:12.547841 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 929/1873] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Sep 6 04:36:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 250 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 04:36:12.549671 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf27032-0542-41c0-9e90-2fed38c14a1c service neutron] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:36:12.563765 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf27032-0542-41c0-9e90-2fed38c14a1c service neutron] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:36:12.567691 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169ff139-939f-40c0-90b9-6a012537188a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:36:12.568403 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169ff139-939f-40c0-90b9-6a012537188a admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:36:12.568667 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169ff139-939f-40c0-90b9-6a012537188a admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:36:12.568941 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169ff139-939f-40c0-90b9-6a012537188a admin admin] PATH_INFO: `/v3/domains` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:36:12.569650 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-169ff139-939f-40c0-90b9-6a012537188a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:36:12.569985 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-169ff139-939f-40c0-90b9-6a012537188a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:36:12.573025 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-169ff139-939f-40c0-90b9-6a012537188a admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:36:12.578260 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 945/1874] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Sep 6 04:36:12 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 04:36:12.583651 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6774952-9832-4ca6-95d2-ad31034bee07 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:36:12.593954 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 930/1875] 10.222.0.30 () {58 vars in 1153 bytes} [Tue Sep 6 04:36:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:36:12.603523 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169ff139-939f-40c0-90b9-6a012537188a admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:36:12.608672 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6774952-9832-4ca6-95d2-ad31034bee07 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:36:12.617793 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe35ea2c-11a5-4fd6-b558-eb519ed5a879 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:36:12.618488 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe35ea2c-11a5-4fd6-b558-eb519ed5a879 admin admin] REQUEST_METHOD: `PUT` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:36:12.618824 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe35ea2c-11a5-4fd6-b558-eb519ed5a879 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:36:12.619107 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe35ea2c-11a5-4fd6-b558-eb519ed5a879 admin admin] PATH_INFO: `/v3/projects/f4d8325f73d44d6db02cc9e937e3f218/users/088224d4ba6c4dc599f1c184d534f089/roles/ea4c814713b149108e488de855b025e4` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:36:12.622499 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe35ea2c-11a5-4fd6-b558-eb519ed5a879 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f4d8325f73d44d6db02cc9e937e3f218, user_id=088224d4ba6c4dc599f1c184d534f089, role_id=ea4c814713b149108e488de855b025e4)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:36:12.624225 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169ff139-939f-40c0-90b9-6a012537188a admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:36:12.626807 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe35ea2c-11a5-4fd6-b558-eb519ed5a879 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:36:12.641152 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 946/1876] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Sep 6 04:36:12 2022] PUT /identity/v3/projects/f4d8325f73d44d6db02cc9e937e3f218/users/088224d4ba6c4dc599f1c184d534f089/roles/ea4c814713b149108e488de855b025e4 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 04:36:12.652261 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b54ffc-4660-49a0-8122-649a00fcc85a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:36:12.653799 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbaaeec-7214-4813-8a3e-69bdcfd68ab5 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:36:12.654092 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbaaeec-7214-4813-8a3e-69bdcfd68ab5 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:36:12.654367 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbaaeec-7214-4813-8a3e-69bdcfd68ab5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:36:12.655982 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b54ffc-4660-49a0-8122-649a00fcc85a admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:36:12.657611 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b54ffc-4660-49a0-8122-649a00fcc85a admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:36:12.657903 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b54ffc-4660-49a0-8122-649a00fcc85a admin admin] PATH_INFO: `/v3/projects` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:36:12.658975 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b54ffc-4660-49a0-8122-649a00fcc85a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:36:12.663253 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b54ffc-4660-49a0-8122-649a00fcc85a admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:36:12.665214 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:36:12.714967 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-2dbaaeec-7214-4813-8a3e-69bdcfd68ab5 None None] MFA Rules not processed for user `088224d4ba6c4dc599f1c184d534f089`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:36:12.716203 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-2dbaaeec-7214-4813-8a3e-69bdcfd68ab5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:36:12.799008 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 947/1877] 10.222.0.30 () {58 vars in 958 bytes} [Tue Sep 6 04:36:12 2022] POST /identity/v3/auth/tokens => generated 3351 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 04:36:12.807727 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dbaaeec-7214-4813-8a3e-69bdcfd68ab5 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:36:12.809409 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-2dbaaeec-7214-4813-8a3e-69bdcfd68ab5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:36:12.826386 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 931/1878] 10.222.0.30 () {60 vars in 1153 bytes} [Tue Sep 6 04:36:12 2022] POST /identity/v3/projects => generated 375 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 04:36:12.833148 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b54ffc-4660-49a0-8122-649a00fcc85a admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:36:12.873904 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dbaaeec-7214-4813-8a3e-69bdcfd68ab5 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:36:12.944609 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b54ffc-4660-49a0-8122-649a00fcc85a admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:36:12.953432 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78304f16-c51e-4694-aff2-59b5060e3749 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:36:12.954643 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78304f16-c51e-4694-aff2-59b5060e3749 admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:36:12.954929 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78304f16-c51e-4694-aff2-59b5060e3749 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:36:12.956117 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78304f16-c51e-4694-aff2-59b5060e3749 admin admin] PATH_INFO: `/v3/users` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:36:12.958325 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78304f16-c51e-4694-aff2-59b5060e3749 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:36:12.961486 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78304f16-c51e-4694-aff2-59b5060e3749 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:36:12.964658 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92db476-6bf6-48e1-87bf-f34317483202 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd09b5ab675634a2eb4b03755743f7d0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b707733ce65c42edbc1fd95ad0a1c06c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:36:12.965347 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92db476-6bf6-48e1-87bf-f34317483202 service cinder] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:36:12.965841 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92db476-6bf6-48e1-87bf-f34317483202 service cinder] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:36:12.965945 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92db476-6bf6-48e1-87bf-f34317483202 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:36:13.043521 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 932/1879] 10.222.0.30 () {60 vars in 1144 bytes} [Tue Sep 6 04:36:12 2022] POST /identity/v3/users => generated 288 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 04:36:13.045673 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92db476-6bf6-48e1-87bf-f34317483202 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:36:13.047439 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78304f16-c51e-4694-aff2-59b5060e3749 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:36:13.048295 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92db476-6bf6-48e1-87bf-f34317483202 service cinder] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:36:13.058091 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78304f16-c51e-4694-aff2-59b5060e3749 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:36:13.062743 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19761cc4-d9b0-4ff3-b131-d56a247dd3e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:36:13.071262 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19761cc4-d9b0-4ff3-b131-d56a247dd3e2 admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:36:13.071554 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 948/1880] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Sep 6 04:36:12 2022] GET /identity/v3/auth/tokens => generated 3351 bytes in 265 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 06 04:36:13.071658 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19761cc4-d9b0-4ff3-b131-d56a247dd3e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:36:13.071835 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19761cc4-d9b0-4ff3-b131-d56a247dd3e2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:36:13.072601 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19761cc4-d9b0-4ff3-b131-d56a247dd3e2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:36:13.074373 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19761cc4-d9b0-4ff3-b131-d56a247dd3e2 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:36:13.091381 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 933/1881] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Sep 6 04:36:13 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 06 04:36:13.092452 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92db476-6bf6-48e1-87bf-f34317483202 service cinder] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:36:13.104493 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92db476-6bf6-48e1-87bf-f34317483202 service cinder] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:36:13.110908 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282d1ef0-dd6e-4024-bb45-b6943bab6125 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:36:13.111538 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282d1ef0-dd6e-4024-bb45-b6943bab6125 admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:36:13.111827 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282d1ef0-dd6e-4024-bb45-b6943bab6125 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:36:13.112261 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282d1ef0-dd6e-4024-bb45-b6943bab6125 admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:36:13.112916 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282d1ef0-dd6e-4024-bb45-b6943bab6125 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:36:13.114782 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282d1ef0-dd6e-4024-bb45-b6943bab6125 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:36:13.129558 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 949/1882] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Sep 6 04:36:13 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 06 04:36:13.131042 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19761cc4-d9b0-4ff3-b131-d56a247dd3e2 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:36:13.142376 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19761cc4-d9b0-4ff3-b131-d56a247dd3e2 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:36:13.147442 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b734a92-cb61-420d-bb7d-bd9b06c16baf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:36:13.148150 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b734a92-cb61-420d-bb7d-bd9b06c16baf admin admin] REQUEST_METHOD: `PUT` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:36:13.148432 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b734a92-cb61-420d-bb7d-bd9b06c16baf admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:36:13.148749 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b734a92-cb61-420d-bb7d-bd9b06c16baf admin admin] PATH_INFO: `/v3/projects/58d6087b1ad045129117eee56c8d019a/users/5936550003094273890e300567f5cb04/roles/29607b2de39f40d68bb6ceef64b8a732` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:36:13.168908 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b734a92-cb61-420d-bb7d-bd9b06c16baf admin admin] RBAC: Authorizing `identity:create_grant(project_id=58d6087b1ad045129117eee56c8d019a, user_id=5936550003094273890e300567f5cb04, role_id=29607b2de39f40d68bb6ceef64b8a732)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:36:13.171495 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b734a92-cb61-420d-bb7d-bd9b06c16baf admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:36:13.259459 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 934/1883] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Sep 6 04:36:13 2022] PUT /identity/v3/projects/58d6087b1ad045129117eee56c8d019a/users/5936550003094273890e300567f5cb04/roles/29607b2de39f40d68bb6ceef64b8a732 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 04:36:13.269913 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa32a56a-450b-4b4e-bdaf-16e23de63994 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:36:13.270421 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa32a56a-450b-4b4e-bdaf-16e23de63994 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:36:13.270840 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa32a56a-450b-4b4e-bdaf-16e23de63994 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:36:13.296558 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b734a92-cb61-420d-bb7d-bd9b06c16baf admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:36:13.298716 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-5b734a92-cb61-420d-bb7d-bd9b06c16baf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:36:13.316166 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-aa32a56a-450b-4b4e-bdaf-16e23de63994 None None] MFA Rules not processed for user `5936550003094273890e300567f5cb04`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:36:13.317747 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-aa32a56a-450b-4b4e-bdaf-16e23de63994 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:36:13.457529 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b734a92-cb61-420d-bb7d-bd9b06c16baf admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:36:13.463680 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b70fe55-eb22-4fa1-9c09-391bb5a25675 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:36:13.464296 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b70fe55-eb22-4fa1-9c09-391bb5a25675 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:36:13.464830 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b70fe55-eb22-4fa1-9c09-391bb5a25675 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:36:13.465744 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b70fe55-eb22-4fa1-9c09-391bb5a25675 admin admin] PATH_INFO: `/v3/projects/8118794c3b26418c92de08614b25d685` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:36:13.467066 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b70fe55-eb22-4fa1-9c09-391bb5a25675 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8118794c3b26418c92de08614b25d685)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:36:13.468970 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b70fe55-eb22-4fa1-9c09-391bb5a25675 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:36:13.494481 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 950/1884] 10.222.0.30 () {58 vars in 958 bytes} [Tue Sep 6 04:36:13 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 04:36:13.496848 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa32a56a-450b-4b4e-bdaf-16e23de63994 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:36:13.524692 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa32a56a-450b-4b4e-bdaf-16e23de63994 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:36:13.536172 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c33b3a0-6283-4096-9c1f-23372e4cb4b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:36:13.536805 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c33b3a0-6283-4096-9c1f-23372e4cb4b9 admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:36:13.537084 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c33b3a0-6283-4096-9c1f-23372e4cb4b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:36:13.537555 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c33b3a0-6283-4096-9c1f-23372e4cb4b9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:36:13.538357 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c33b3a0-6283-4096-9c1f-23372e4cb4b9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:36:13.540683 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c33b3a0-6283-4096-9c1f-23372e4cb4b9 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:36:13.622379 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-3b70fe55-eb22-4fa1-9c09-391bb5a25675 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8118794c3b26418c92de08614b25d685', 'request_id': 'req-3b70fe55-eb22-4fa1-9c09-391bb5a25675'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:36:13.647107 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 951/1885] 10.222.0.30 () {60 vars in 1153 bytes} [Tue Sep 6 04:36:13 2022] POST /identity/v3/projects => generated 381 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 04:36:13.648406 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c33b3a0-6283-4096-9c1f-23372e4cb4b9 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:36:13.650941 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-4c33b3a0-6283-4096-9c1f-23372e4cb4b9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:36:13.699280 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-3b70fe55-eb22-4fa1-9c09-391bb5a25675 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8118794c3b26418c92de08614b25d685', 'request_id': 'req-3b70fe55-eb22-4fa1-9c09-391bb5a25675'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:36:13.713734 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-3b70fe55-eb22-4fa1-9c09-391bb5a25675 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8118794c3b26418c92de08614b25d685', 'request_id': 'req-3b70fe55-eb22-4fa1-9c09-391bb5a25675'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:36:13.713905 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c33b3a0-6283-4096-9c1f-23372e4cb4b9 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:36:13.714064 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb4611f-3404-432c-8be9-1de71dd71310 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd09b5ab675634a2eb4b03755743f7d0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b707733ce65c42edbc1fd95ad0a1c06c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:36:13.714786 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb4611f-3404-432c-8be9-1de71dd71310 service cinder] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:36:13.715072 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb4611f-3404-432c-8be9-1de71dd71310 service cinder] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:36:13.715414 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb4611f-3404-432c-8be9-1de71dd71310 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:36:13.717159 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-3cb4611f-3404-432c-8be9-1de71dd71310 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:36:13.718941 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-3b70fe55-eb22-4fa1-9c09-391bb5a25675 admin admin] The token cache is being invalidate because project 8118794c3b26418c92de08614b25d685 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81697) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 04:36:13.719286 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-3b70fe55-eb22-4fa1-9c09-391bb5a25675 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:36:13.719931 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-3b70fe55-eb22-4fa1-9c09-391bb5a25675 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:36:13.724544 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 935/1886] 10.222.0.30 () {58 vars in 1233 bytes} [Tue Sep 6 04:36:13 2022] DELETE /identity/v3/projects/8118794c3b26418c92de08614b25d685 => generated 0 bytes in 426 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 04:36:13.724544 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b70fe55-eb22-4fa1-9c09-391bb5a25675 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:36:13.725900 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-3b70fe55-eb22-4fa1-9c09-391bb5a25675 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:36:13.782700 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb4611f-3404-432c-8be9-1de71dd71310 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:36:13.784247 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb4611f-3404-432c-8be9-1de71dd71310 service cinder] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:36:13.794733 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 952/1887] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Sep 6 04:36:13 2022] GET /identity/v3/auth/tokens => generated 3286 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 04:36:13.795580 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb4611f-3404-432c-8be9-1de71dd71310 service cinder] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:36:13.803908 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:36:13.809976 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b70fe55-eb22-4fa1-9c09-391bb5a25675 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:36:13.876103 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb4611f-3404-432c-8be9-1de71dd71310 service cinder] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:36:13.880307 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5bf486f-1c2c-4cdb-9b9f-eecaa280fd44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:36:13.880976 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d966c59-9f12-436a-b157-3dc998095092 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:36:13.881454 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bf486f-1c2c-4cdb-9b9f-eecaa280fd44 admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:36:13.881648 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d966c59-9f12-436a-b157-3dc998095092 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:36:13.881749 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bf486f-1c2c-4cdb-9b9f-eecaa280fd44 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:36:13.881918 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d966c59-9f12-436a-b157-3dc998095092 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:36:13.882023 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bf486f-1c2c-4cdb-9b9f-eecaa280fd44 admin admin] PATH_INFO: `/v3/users` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:36:13.882185 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d966c59-9f12-436a-b157-3dc998095092 admin admin] PATH_INFO: `/v3/users/daeab80654c3421687cdcaeef38bc63e` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:36:13.882703 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5bf486f-1c2c-4cdb-9b9f-eecaa280fd44 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:36:13.883311 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d966c59-9f12-436a-b157-3dc998095092 admin admin] RBAC: Authorizing `identity:delete_user(user_id=daeab80654c3421687cdcaeef38bc63e)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:36:13.884683 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5bf486f-1c2c-4cdb-9b9f-eecaa280fd44 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:36:13.885259 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d966c59-9f12-436a-b157-3dc998095092 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:36:13.967547 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 936/1888] 10.222.0.30 () {60 vars in 1144 bytes} [Tue Sep 6 04:36:13 2022] POST /identity/v3/users => generated 297 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 04:36:13.971542 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5bf486f-1c2c-4cdb-9b9f-eecaa280fd44 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:36:13.982309 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5bf486f-1c2c-4cdb-9b9f-eecaa280fd44 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:36:13.987079 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf698fa7-9305-411e-8596-4e1135da68be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:36:13.987684 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf698fa7-9305-411e-8596-4e1135da68be admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:36:13.987966 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf698fa7-9305-411e-8596-4e1135da68be admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:36:13.988317 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf698fa7-9305-411e-8596-4e1135da68be admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:36:13.991512 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf698fa7-9305-411e-8596-4e1135da68be admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:36:13.993115 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf698fa7-9305-411e-8596-4e1135da68be admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:36:14.004685 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 937/1889] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Sep 6 04:36:13 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 06 04:36:14.008567 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf698fa7-9305-411e-8596-4e1135da68be admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:36:14.018626 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf698fa7-9305-411e-8596-4e1135da68be admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:36:14.023240 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11be057-1ff9-42d3-b3b7-a39b520e995d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:36:14.023891 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11be057-1ff9-42d3-b3b7-a39b520e995d admin admin] REQUEST_METHOD: `PUT` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:36:14.024455 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11be057-1ff9-42d3-b3b7-a39b520e995d admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:36:14.024706 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11be057-1ff9-42d3-b3b7-a39b520e995d admin admin] PATH_INFO: `/v3/projects/3a0efc68c5194d3f83abe71ef9b89031/users/6b4be616999a4213a63126e082cda27d/roles/ea4c814713b149108e488de855b025e4` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:36:14.040197 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a11be057-1ff9-42d3-b3b7-a39b520e995d admin admin] RBAC: Authorizing `identity:create_grant(project_id=3a0efc68c5194d3f83abe71ef9b89031, user_id=6b4be616999a4213a63126e082cda27d, role_id=ea4c814713b149108e488de855b025e4)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:36:14.043887 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a11be057-1ff9-42d3-b3b7-a39b520e995d admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:36:14.098403 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-4d966c59-9f12-436a-b157-3dc998095092 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'daeab80654c3421687cdcaeef38bc63e'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:36:14.147697 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 938/1890] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Sep 6 04:36:14 2022] PUT /identity/v3/projects/3a0efc68c5194d3f83abe71ef9b89031/users/6b4be616999a4213a63126e082cda27d/roles/ea4c814713b149108e488de855b025e4 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 04:36:14.155988 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11be057-1ff9-42d3-b3b7-a39b520e995d admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:36:14.172980 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-4d966c59-9f12-436a-b157-3dc998095092 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'daeab80654c3421687cdcaeef38bc63e'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:36:14.173614 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-4d966c59-9f12-436a-b157-3dc998095092 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'daeab80654c3421687cdcaeef38bc63e'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:36:14.191825 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-4d966c59-9f12-436a-b157-3dc998095092 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'daeab80654c3421687cdcaeef38bc63e'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:36:14.202244 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-4d966c59-9f12-436a-b157-3dc998095092 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'daeab80654c3421687cdcaeef38bc63e'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:36:14.210706 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 953/1891] 10.222.0.30 () {58 vars in 1224 bytes} [Tue Sep 6 04:36:13 2022] DELETE /identity/v3/users/daeab80654c3421687cdcaeef38bc63e => generated 0 bytes in 415 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 04:36:14.237542 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11be057-1ff9-42d3-b3b7-a39b520e995d admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:36:14.313712 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75a98aa-fcfd-48aa-9ead-de2c93910575 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:36:14.314463 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75a98aa-fcfd-48aa-9ead-de2c93910575 admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:36:14.314817 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75a98aa-fcfd-48aa-9ead-de2c93910575 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:36:14.316319 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75a98aa-fcfd-48aa-9ead-de2c93910575 admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:36:14.317068 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b75a98aa-fcfd-48aa-9ead-de2c93910575 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:36:14.318728 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b75a98aa-fcfd-48aa-9ead-de2c93910575 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:36:14.330516 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 939/1892] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Sep 6 04:36:14 2022] GET /identity/v3/roles => generated 1420 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 06 04:36:14.334741 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d966c59-9f12-436a-b157-3dc998095092 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:36:14.336209 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-4d966c59-9f12-436a-b157-3dc998095092 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:36:14.350123 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d966c59-9f12-436a-b157-3dc998095092 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:36:14.355654 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7ab9dc-be99-4f9d-9d9d-25d79a7adcb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:36:14.356356 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7ab9dc-be99-4f9d-9d9d-25d79a7adcb7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:36:14.356762 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7ab9dc-be99-4f9d-9d9d-25d79a7adcb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:36:14.357338 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7ab9dc-be99-4f9d-9d9d-25d79a7adcb7 admin admin] PATH_INFO: `/v3/projects/3a0efc68c5194d3f83abe71ef9b89031/users/6b4be616999a4213a63126e082cda27d/roles/ea4c814713b149108e488de855b025e4` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:36:14.359355 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7ab9dc-be99-4f9d-9d9d-25d79a7adcb7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3a0efc68c5194d3f83abe71ef9b89031, user_id=6b4be616999a4213a63126e082cda27d, role_id=ea4c814713b149108e488de855b025e4)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:36:14.362494 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7ab9dc-be99-4f9d-9d9d-25d79a7adcb7 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:36:14.418443 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 954/1893] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Sep 6 04:36:14 2022] PUT /identity/v3/projects/3a0efc68c5194d3f83abe71ef9b89031/users/6b4be616999a4213a63126e082cda27d/roles/ea4c814713b149108e488de855b025e4 => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 04:36:14.441843 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e442e65-62c3-4aec-bd5b-a35113519980 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:36:14.442204 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e442e65-62c3-4aec-bd5b-a35113519980 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:36:14.442562 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e442e65-62c3-4aec-bd5b-a35113519980 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:36:14.453201 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:36:14.499557 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-4e442e65-62c3-4aec-bd5b-a35113519980 None None] MFA Rules not processed for user `6b4be616999a4213a63126e082cda27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:36:14.499557 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-4e442e65-62c3-4aec-bd5b-a35113519980 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:36:14.628563 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 940/1894] 10.222.0.30 () {58 vars in 958 bytes} [Tue Sep 6 04:36:14 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 04:36:14.652296 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7ab9dc-be99-4f9d-9d9d-25d79a7adcb7 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:36:14.654015 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-0b7ab9dc-be99-4f9d-9d9d-25d79a7adcb7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:36:14.809841 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7ab9dc-be99-4f9d-9d9d-25d79a7adcb7 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:36:14.810402 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e442e65-62c3-4aec-bd5b-a35113519980 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:36:14.812486 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-4e442e65-62c3-4aec-bd5b-a35113519980 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:36:14.813948 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f81f8ee-b114-42fc-b2ef-213f5fed30c8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd09b5ab675634a2eb4b03755743f7d0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b707733ce65c42edbc1fd95ad0a1c06c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:36:14.814650 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f81f8ee-b114-42fc-b2ef-213f5fed30c8 service cinder] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:36:14.815067 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f81f8ee-b114-42fc-b2ef-213f5fed30c8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:36:14.815325 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f81f8ee-b114-42fc-b2ef-213f5fed30c8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:36:14.837701 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f81f8ee-b114-42fc-b2ef-213f5fed30c8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:36:14.840101 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f81f8ee-b114-42fc-b2ef-213f5fed30c8 service cinder] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:36:14.852332 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 955/1895] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Sep 6 04:36:14 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 201 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 04:36:14.929189 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e442e65-62c3-4aec-bd5b-a35113519980 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:36:14.934904 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d8dd7c-f10b-47b6-bb4c-f70abb27ed92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:36:14.937275 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d8dd7c-f10b-47b6-bb4c-f70abb27ed92 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:36:14.937275 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d8dd7c-f10b-47b6-bb4c-f70abb27ed92 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:36:14.937275 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d8dd7c-f10b-47b6-bb4c-f70abb27ed92 admin admin] PATH_INFO: `/v3/projects/9d9dd6c3c81e4506a5c198e670c461fa` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:36:14.937548 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d8dd7c-f10b-47b6-bb4c-f70abb27ed92 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9d9dd6c3c81e4506a5c198e670c461fa)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:36:14.939293 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d8dd7c-f10b-47b6-bb4c-f70abb27ed92 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:36:15.102185 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-b4d8dd7c-f10b-47b6-bb4c-f70abb27ed92 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9d9dd6c3c81e4506a5c198e670c461fa', 'request_id': 'req-b4d8dd7c-f10b-47b6-bb4c-f70abb27ed92'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:36:15.182933 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-b4d8dd7c-f10b-47b6-bb4c-f70abb27ed92 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9d9dd6c3c81e4506a5c198e670c461fa', 'request_id': 'req-b4d8dd7c-f10b-47b6-bb4c-f70abb27ed92'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:36:15.190216 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-b4d8dd7c-f10b-47b6-bb4c-f70abb27ed92 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9d9dd6c3c81e4506a5c198e670c461fa', 'request_id': 'req-b4d8dd7c-f10b-47b6-bb4c-f70abb27ed92'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:36:15.201419 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-b4d8dd7c-f10b-47b6-bb4c-f70abb27ed92 admin admin] The token cache is being invalidate because project 9d9dd6c3c81e4506a5c198e670c461fa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81697) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 04:36:15.201714 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-b4d8dd7c-f10b-47b6-bb4c-f70abb27ed92 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:36:15.202438 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-b4d8dd7c-f10b-47b6-bb4c-f70abb27ed92 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:36:15.204144 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 941/1896] 10.222.0.30 () {58 vars in 1233 bytes} [Tue Sep 6 04:36:14 2022] DELETE /identity/v3/projects/9d9dd6c3c81e4506a5c198e670c461fa => generated 0 bytes in 395 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 04:36:15.208814 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a5ca3c-4a33-45e6-94b2-904f63716e01 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:36:15.209270 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a5ca3c-4a33-45e6-94b2-904f63716e01 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:36:15.209602 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a5ca3c-4a33-45e6-94b2-904f63716e01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:36:15.239064 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-02a5ca3c-4a33-45e6-94b2-904f63716e01 None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:36:15.240346 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-02a5ca3c-4a33-45e6-94b2-904f63716e01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:36:15.310180 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 956/1897] 10.222.0.30 () {58 vars in 958 bytes} [Tue Sep 6 04:36:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 04:36:15.317939 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585357a3-b75a-4977-895e-f027b490b6c2 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:36:15.318223 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585357a3-b75a-4977-895e-f027b490b6c2 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:36:15.318499 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585357a3-b75a-4977-895e-f027b490b6c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:36:15.343159 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-585357a3-b75a-4977-895e-f027b490b6c2 None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:36:15.343159 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-585357a3-b75a-4977-895e-f027b490b6c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:36:15.365359 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 942/1898] 10.222.0.30 () {58 vars in 958 bytes} [Tue Sep 6 04:36:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 04:36:15.374011 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a5ca3c-4a33-45e6-94b2-904f63716e01 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:36:15.386441 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a5ca3c-4a33-45e6-94b2-904f63716e01 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:36:15.396363 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb39898-fb7e-4ed3-9b6b-b3034645aecc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:36:15.401416 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb39898-fb7e-4ed3-9b6b-b3034645aecc admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:36:15.401416 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb39898-fb7e-4ed3-9b6b-b3034645aecc admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:36:15.401416 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb39898-fb7e-4ed3-9b6b-b3034645aecc admin admin] PATH_INFO: `/v3/domains` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:36:15.401416 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb39898-fb7e-4ed3-9b6b-b3034645aecc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:36:15.401416 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb39898-fb7e-4ed3-9b6b-b3034645aecc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:36:15.401416 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb39898-fb7e-4ed3-9b6b-b3034645aecc admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:36:15.419689 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 957/1899] 10.222.0.30 () {58 vars in 1153 bytes} [Tue Sep 6 04:36:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:36:15.420888 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585357a3-b75a-4977-895e-f027b490b6c2 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:36:15.435800 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585357a3-b75a-4977-895e-f027b490b6c2 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:36:15.438428 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cca11dc-4f7d-4aef-b398-7a8f8e27337f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:36:15.439548 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cca11dc-4f7d-4aef-b398-7a8f8e27337f admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:36:15.439904 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cca11dc-4f7d-4aef-b398-7a8f8e27337f admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:36:15.440306 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cca11dc-4f7d-4aef-b398-7a8f8e27337f admin admin] PATH_INFO: `/v3/projects` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:36:15.440615 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb39898-fb7e-4ed3-9b6b-b3034645aecc admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:36:15.440873 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cca11dc-4f7d-4aef-b398-7a8f8e27337f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:36:15.442533 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-0cb39898-fb7e-4ed3-9b6b-b3034645aecc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:36:15.443108 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cca11dc-4f7d-4aef-b398-7a8f8e27337f admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:36:15.459809 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb39898-fb7e-4ed3-9b6b-b3034645aecc admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:36:15.467353 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a4b77c-dd31-40e0-8591-65e4b686451e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:36:15.469920 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a4b77c-dd31-40e0-8591-65e4b686451e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:36:15.469920 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a4b77c-dd31-40e0-8591-65e4b686451e admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:36:15.471466 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a4b77c-dd31-40e0-8591-65e4b686451e admin admin] PATH_INFO: `/v3/users/5936550003094273890e300567f5cb04` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:36:15.473884 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:36:15.475468 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a4b77c-dd31-40e0-8591-65e4b686451e admin admin] RBAC: Authorizing `identity:delete_user(user_id=5936550003094273890e300567f5cb04)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:36:15.477620 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a4b77c-dd31-40e0-8591-65e4b686451e admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:36:15.553342 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 943/1900] 10.222.0.30 () {60 vars in 1153 bytes} [Tue Sep 6 04:36:15 2022] POST /identity/v3/projects => generated 385 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 04:36:15.554454 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cca11dc-4f7d-4aef-b398-7a8f8e27337f admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:36:15.638448 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cca11dc-4f7d-4aef-b398-7a8f8e27337f admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:36:15.643617 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc33457-4019-4d2a-ad37-cddec857f938 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:36:15.644241 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc33457-4019-4d2a-ad37-cddec857f938 admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:36:15.644520 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc33457-4019-4d2a-ad37-cddec857f938 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:36:15.644792 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc33457-4019-4d2a-ad37-cddec857f938 admin admin] PATH_INFO: `/v3/users` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:36:15.645644 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc33457-4019-4d2a-ad37-cddec857f938 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:36:15.652356 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc33457-4019-4d2a-ad37-cddec857f938 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:36:15.725246 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-e9a4b77c-dd31-40e0-8591-65e4b686451e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5936550003094273890e300567f5cb04'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:36:15.738973 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:36:15.745227 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 944/1901] 10.222.0.30 () {60 vars in 1144 bytes} [Tue Sep 6 04:36:15 2022] POST /identity/v3/users => generated 293 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 04:36:15.749513 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc33457-4019-4d2a-ad37-cddec857f938 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:36:15.759011 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc33457-4019-4d2a-ad37-cddec857f938 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:36:15.764360 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea3a689-3482-4f80-991a-95cdefca1973 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:36:15.764952 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea3a689-3482-4f80-991a-95cdefca1973 admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:36:15.765237 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea3a689-3482-4f80-991a-95cdefca1973 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:36:15.765543 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea3a689-3482-4f80-991a-95cdefca1973 admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:36:15.766237 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea3a689-3482-4f80-991a-95cdefca1973 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:36:15.767815 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea3a689-3482-4f80-991a-95cdefca1973 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:36:15.779233 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 945/1902] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Sep 6 04:36:15 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 06 04:36:15.783374 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea3a689-3482-4f80-991a-95cdefca1973 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:36:15.794032 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea3a689-3482-4f80-991a-95cdefca1973 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:36:15.798684 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85406095-e4a4-4362-bc9b-3fe2086ecb78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:36:15.799279 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85406095-e4a4-4362-bc9b-3fe2086ecb78 admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:36:15.799558 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85406095-e4a4-4362-bc9b-3fe2086ecb78 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:36:15.799831 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85406095-e4a4-4362-bc9b-3fe2086ecb78 admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:36:15.800464 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85406095-e4a4-4362-bc9b-3fe2086ecb78 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:36:15.802088 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85406095-e4a4-4362-bc9b-3fe2086ecb78 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:36:15.813455 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 946/1903] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Sep 6 04:36:15 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 06 04:36:15.816294 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-e9a4b77c-dd31-40e0-8591-65e4b686451e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5936550003094273890e300567f5cb04'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:36:15.817085 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-e9a4b77c-dd31-40e0-8591-65e4b686451e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5936550003094273890e300567f5cb04'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:36:15.817751 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85406095-e4a4-4362-bc9b-3fe2086ecb78 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:36:15.829154 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85406095-e4a4-4362-bc9b-3fe2086ecb78 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:36:15.830862 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-e9a4b77c-dd31-40e0-8591-65e4b686451e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5936550003094273890e300567f5cb04'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:36:15.833857 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5266d8-b1b0-4e56-bf8f-ac1f6bf518e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:36:15.834508 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5266d8-b1b0-4e56-bf8f-ac1f6bf518e0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:36:15.834787 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5266d8-b1b0-4e56-bf8f-ac1f6bf518e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:36:15.835057 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5266d8-b1b0-4e56-bf8f-ac1f6bf518e0 admin admin] PATH_INFO: `/v3/projects/e4ffc44366814a1eaaf99740e6e66808/users/94407dcea4ea4222852b656fc1c7a20b/roles/29607b2de39f40d68bb6ceef64b8a732` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:36:15.837935 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-e9a4b77c-dd31-40e0-8591-65e4b686451e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5936550003094273890e300567f5cb04'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:36:15.844929 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 958/1904] 10.222.0.30 () {58 vars in 1224 bytes} [Tue Sep 6 04:36:15 2022] DELETE /identity/v3/users/5936550003094273890e300567f5cb04 => generated 0 bytes in 405 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 04:36:15.849249 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85550de-3673-43a2-92a6-3d5da2ec5661 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:36:15.849549 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85550de-3673-43a2-92a6-3d5da2ec5661 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:36:15.849832 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85550de-3673-43a2-92a6-3d5da2ec5661 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:36:15.852773 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5266d8-b1b0-4e56-bf8f-ac1f6bf518e0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e4ffc44366814a1eaaf99740e6e66808, user_id=94407dcea4ea4222852b656fc1c7a20b, role_id=29607b2de39f40d68bb6ceef64b8a732)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:36:15.855454 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5266d8-b1b0-4e56-bf8f-ac1f6bf518e0 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:36:15.868941 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-d85550de-3673-43a2-92a6-3d5da2ec5661 None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:36:15.870736 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-d85550de-3673-43a2-92a6-3d5da2ec5661 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:36:15.927453 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 959/1905] 10.222.0.30 () {58 vars in 958 bytes} [Tue Sep 6 04:36:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 04:36:15.937583 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85550de-3673-43a2-92a6-3d5da2ec5661 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:36:15.938912 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-d85550de-3673-43a2-92a6-3d5da2ec5661 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:36:15.962246 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 947/1906] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Sep 6 04:36:15 2022] PUT /identity/v3/projects/e4ffc44366814a1eaaf99740e6e66808/users/94407dcea4ea4222852b656fc1c7a20b/roles/29607b2de39f40d68bb6ceef64b8a732 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 04:36:15.972693 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da0d5ba-1e73-41cb-9469-b5a978a9659c None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:36:15.976173 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da0d5ba-1e73-41cb-9469-b5a978a9659c None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:36:15.976173 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da0d5ba-1e73-41cb-9469-b5a978a9659c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:36:16.000363 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85550de-3673-43a2-92a6-3d5da2ec5661 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:36:16.004507 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-5da0d5ba-1e73-41cb-9469-b5a978a9659c None None] MFA Rules not processed for user `94407dcea4ea4222852b656fc1c7a20b`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:36:16.008813 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-5da0d5ba-1e73-41cb-9469-b5a978a9659c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:36:16.057474 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09d17ab-3ffc-4807-891b-5602c16f2d8a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90492440f6a6471e8392361c098428aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b707733ce65c42edbc1fd95ad0a1c06c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:36:16.058180 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09d17ab-3ffc-4807-891b-5602c16f2d8a service neutron] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:36:16.058463 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09d17ab-3ffc-4807-891b-5602c16f2d8a service neutron] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:36:16.059141 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09d17ab-3ffc-4807-891b-5602c16f2d8a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:36:16.076253 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 948/1907] 10.222.0.30 () {58 vars in 958 bytes} [Tue Sep 6 04:36:15 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 04:36:16.080980 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da0d5ba-1e73-41cb-9469-b5a978a9659c None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:36:16.082486 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-5da0d5ba-1e73-41cb-9469-b5a978a9659c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:36:16.127885 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09d17ab-3ffc-4807-891b-5602c16f2d8a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:36:16.129531 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09d17ab-3ffc-4807-891b-5602c16f2d8a service neutron] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:36:16.139846 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 960/1908] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Sep 6 04:36:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 203 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 06 04:36:16.164076 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da0d5ba-1e73-41cb-9469-b5a978a9659c None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:36:16.169823 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43bf8fb7-2f50-4f2b-9162-3e929d55ce77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:36:16.170989 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bf8fb7-2f50-4f2b-9162-3e929d55ce77 admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:36:16.170989 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bf8fb7-2f50-4f2b-9162-3e929d55ce77 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:36:16.171222 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bf8fb7-2f50-4f2b-9162-3e929d55ce77 admin admin] PATH_INFO: `/v3/projects` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:36:16.171867 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43bf8fb7-2f50-4f2b-9162-3e929d55ce77 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:36:16.174005 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43bf8fb7-2f50-4f2b-9162-3e929d55ce77 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:36:16.310382 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 949/1909] 10.222.0.30 () {60 vars in 1153 bytes} [Tue Sep 6 04:36:16 2022] POST /identity/v3/projects => generated 385 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 04:36:16.311519 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09d17ab-3ffc-4807-891b-5602c16f2d8a service neutron] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:36:16.321930 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09d17ab-3ffc-4807-891b-5602c16f2d8a service neutron] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:36:16.383839 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804a4a57-c53a-4482-866e-1b94c976deb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:36:16.384744 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804a4a57-c53a-4482-866e-1b94c976deb6 admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:36:16.385198 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804a4a57-c53a-4482-866e-1b94c976deb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:36:16.385673 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804a4a57-c53a-4482-866e-1b94c976deb6 admin admin] PATH_INFO: `/v3/users` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:36:16.386320 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804a4a57-c53a-4482-866e-1b94c976deb6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:36:16.388274 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804a4a57-c53a-4482-866e-1b94c976deb6 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:36:16.541536 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 961/1910] 10.222.0.30 () {60 vars in 1144 bytes} [Tue Sep 6 04:36:16 2022] POST /identity/v3/users => generated 299 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 04:36:16.541646 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43bf8fb7-2f50-4f2b-9162-3e929d55ce77 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:36:16.551694 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43bf8fb7-2f50-4f2b-9162-3e929d55ce77 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:36:16.557007 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f4a22a-e0c6-4f19-9fd4-d26f359b485f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:36:16.557893 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f4a22a-e0c6-4f19-9fd4-d26f359b485f admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:36:16.558036 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f4a22a-e0c6-4f19-9fd4-d26f359b485f admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:36:16.558317 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f4a22a-e0c6-4f19-9fd4-d26f359b485f admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:36:16.558976 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f4a22a-e0c6-4f19-9fd4-d26f359b485f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:36:16.560767 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f4a22a-e0c6-4f19-9fd4-d26f359b485f admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:36:16.589881 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 950/1911] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Sep 6 04:36:16 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 06 04:36:16.598173 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804a4a57-c53a-4482-866e-1b94c976deb6 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:36:16.621913 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804a4a57-c53a-4482-866e-1b94c976deb6 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:36:16.639226 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b2a78b-19ed-4525-b935-d17460876349 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:36:16.640124 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b2a78b-19ed-4525-b935-d17460876349 admin admin] REQUEST_METHOD: `PUT` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:36:16.640509 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b2a78b-19ed-4525-b935-d17460876349 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:36:16.640873 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b2a78b-19ed-4525-b935-d17460876349 admin admin] PATH_INFO: `/v3/projects/77fde6d9ba394d79adba96ed3d3b7aed/users/b9e73262a85e4956b25be88a9ae2a976/roles/ea4c814713b149108e488de855b025e4` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:36:16.657595 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b2a78b-19ed-4525-b935-d17460876349 admin admin] RBAC: Authorizing `identity:create_grant(project_id=77fde6d9ba394d79adba96ed3d3b7aed, user_id=b9e73262a85e4956b25be88a9ae2a976, role_id=ea4c814713b149108e488de855b025e4)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:36:16.660408 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b2a78b-19ed-4525-b935-d17460876349 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:36:16.721285 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f4a22a-e0c6-4f19-9fd4-d26f359b485f admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:36:16.721285 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-38f4a22a-e0c6-4f19-9fd4-d26f359b485f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:36:16.736044 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f4a22a-e0c6-4f19-9fd4-d26f359b485f admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:36:16.754217 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 962/1912] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Sep 6 04:36:16 2022] PUT /identity/v3/projects/77fde6d9ba394d79adba96ed3d3b7aed/users/b9e73262a85e4956b25be88a9ae2a976/roles/ea4c814713b149108e488de855b025e4 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 04:36:16.755048 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3472a67-671a-4a4b-b0c6-15f0d275f700 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:36:16.755509 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b2a78b-19ed-4525-b935-d17460876349 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:36:16.755730 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3472a67-671a-4a4b-b0c6-15f0d275f700 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:36:16.756315 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3472a67-671a-4a4b-b0c6-15f0d275f700 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:36:16.756676 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3472a67-671a-4a4b-b0c6-15f0d275f700 admin admin] PATH_INFO: `/v3/projects/58d6087b1ad045129117eee56c8d019a` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:36:16.758425 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3472a67-671a-4a4b-b0c6-15f0d275f700 admin admin] RBAC: Authorizing `identity:delete_project(project_id=58d6087b1ad045129117eee56c8d019a)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:36:16.761018 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3472a67-671a-4a4b-b0c6-15f0d275f700 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:36:16.857832 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b2a78b-19ed-4525-b935-d17460876349 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:36:16.862679 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1244f2d-ff87-42cb-9a2e-a564c8b7080f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:36:16.866617 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1244f2d-ff87-42cb-9a2e-a564c8b7080f admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:36:16.866617 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1244f2d-ff87-42cb-9a2e-a564c8b7080f admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:36:16.866617 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1244f2d-ff87-42cb-9a2e-a564c8b7080f admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:36:16.866617 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1244f2d-ff87-42cb-9a2e-a564c8b7080f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:36:16.866617 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1244f2d-ff87-42cb-9a2e-a564c8b7080f admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:36:16.876848 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 963/1913] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Sep 6 04:36:16 2022] GET /identity/v3/roles => generated 1420 bytes in 122 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 04:36:16.881631 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1244f2d-ff87-42cb-9a2e-a564c8b7080f admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:36:16.892691 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1244f2d-ff87-42cb-9a2e-a564c8b7080f admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:36:16.906401 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-e3472a67-671a-4a4b-b0c6-15f0d275f700 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '58d6087b1ad045129117eee56c8d019a', 'request_id': 'req-e3472a67-671a-4a4b-b0c6-15f0d275f700'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:36:16.951600 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34eb2cc6-c1cd-4862-910d-73970924f81c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:36:16.952258 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34eb2cc6-c1cd-4862-910d-73970924f81c admin admin] REQUEST_METHOD: `PUT` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:36:16.952606 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34eb2cc6-c1cd-4862-910d-73970924f81c admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:36:16.952840 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34eb2cc6-c1cd-4862-910d-73970924f81c admin admin] PATH_INFO: `/v3/projects/77fde6d9ba394d79adba96ed3d3b7aed/users/b9e73262a85e4956b25be88a9ae2a976/roles/ea4c814713b149108e488de855b025e4` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:36:16.954764 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34eb2cc6-c1cd-4862-910d-73970924f81c admin admin] RBAC: Authorizing `identity:create_grant(project_id=77fde6d9ba394d79adba96ed3d3b7aed, user_id=b9e73262a85e4956b25be88a9ae2a976, role_id=ea4c814713b149108e488de855b025e4)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:36:16.957502 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34eb2cc6-c1cd-4862-910d-73970924f81c admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:36:16.967965 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 964/1914] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Sep 6 04:36:16 2022] PUT /identity/v3/projects/77fde6d9ba394d79adba96ed3d3b7aed/users/b9e73262a85e4956b25be88a9ae2a976/roles/ea4c814713b149108e488de855b025e4 => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 04:36:16.980165 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a5cfd3-9938-4218-b2ca-aafbf8c64966 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:36:16.980465 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a5cfd3-9938-4218-b2ca-aafbf8c64966 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:36:16.980736 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a5cfd3-9938-4218-b2ca-aafbf8c64966 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:36:16.986137 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-e3472a67-671a-4a4b-b0c6-15f0d275f700 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '58d6087b1ad045129117eee56c8d019a', 'request_id': 'req-e3472a67-671a-4a4b-b0c6-15f0d275f700'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:36:16.996520 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-e3472a67-671a-4a4b-b0c6-15f0d275f700 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '58d6087b1ad045129117eee56c8d019a', 'request_id': 'req-e3472a67-671a-4a4b-b0c6-15f0d275f700'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:36:17.007900 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-e3472a67-671a-4a4b-b0c6-15f0d275f700 admin admin] The token cache is being invalidate because project 58d6087b1ad045129117eee56c8d019a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81697) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 04:36:17.008280 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-e3472a67-671a-4a4b-b0c6-15f0d275f700 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:36:17.009674 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-e3472a67-671a-4a4b-b0c6-15f0d275f700 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:36:17.012011 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 951/1915] 10.222.0.30 () {58 vars in 1233 bytes} [Tue Sep 6 04:36:16 2022] DELETE /identity/v3/projects/58d6087b1ad045129117eee56c8d019a => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 04:36:17.012236 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-f7a5cfd3-9938-4218-b2ca-aafbf8c64966 None None] MFA Rules not processed for user `b9e73262a85e4956b25be88a9ae2a976`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:36:17.013526 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-f7a5cfd3-9938-4218-b2ca-aafbf8c64966 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:36:17.015561 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3472a67-671a-4a4b-b0c6-15f0d275f700 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:36:17.016979 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-e3472a67-671a-4a4b-b0c6-15f0d275f700 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:36:17.032994 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3472a67-671a-4a4b-b0c6-15f0d275f700 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:36:17.071283 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:36:17.097974 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc983524-7d24-4eed-ad75-7b3a586705f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:36:17.098897 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc983524-7d24-4eed-ad75-7b3a586705f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:36:17.100175 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc983524-7d24-4eed-ad75-7b3a586705f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:36:17.100175 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc983524-7d24-4eed-ad75-7b3a586705f5 admin admin] PATH_INFO: `/v3/users/6b4be616999a4213a63126e082cda27d` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:36:17.101795 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 965/1916] 10.222.0.30 () {58 vars in 958 bytes} [Tue Sep 6 04:36:16 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 04:36:17.101912 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc983524-7d24-4eed-ad75-7b3a586705f5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6b4be616999a4213a63126e082cda27d)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:36:17.105156 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc983524-7d24-4eed-ad75-7b3a586705f5 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:36:17.114897 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a5cfd3-9938-4218-b2ca-aafbf8c64966 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:36:17.116279 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-f7a5cfd3-9938-4218-b2ca-aafbf8c64966 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:36:17.198357 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a5cfd3-9938-4218-b2ca-aafbf8c64966 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:36:17.202455 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d8d2d8-8898-4ac5-8aeb-33f7f56cb908 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd09b5ab675634a2eb4b03755743f7d0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b707733ce65c42edbc1fd95ad0a1c06c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:36:17.203567 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d8d2d8-8898-4ac5-8aeb-33f7f56cb908 service cinder] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:36:17.203567 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d8d2d8-8898-4ac5-8aeb-33f7f56cb908 service cinder] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:36:17.203921 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d8d2d8-8898-4ac5-8aeb-33f7f56cb908 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:36:17.206303 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-21d8d2d8-8898-4ac5-8aeb-33f7f56cb908 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:36:17.240751 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:36:17.271368 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d8d2d8-8898-4ac5-8aeb-33f7f56cb908 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:36:17.272965 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d8d2d8-8898-4ac5-8aeb-33f7f56cb908 service cinder] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:36:17.282904 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 966/1917] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Sep 6 04:36:17 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 06 04:36:17.396815 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-dc983524-7d24-4eed-ad75-7b3a586705f5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6b4be616999a4213a63126e082cda27d'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:36:17.489045 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-dc983524-7d24-4eed-ad75-7b3a586705f5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6b4be616999a4213a63126e082cda27d'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:36:17.490841 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-dc983524-7d24-4eed-ad75-7b3a586705f5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6b4be616999a4213a63126e082cda27d'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:36:17.499937 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-dc983524-7d24-4eed-ad75-7b3a586705f5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6b4be616999a4213a63126e082cda27d'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:36:17.519724 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-dc983524-7d24-4eed-ad75-7b3a586705f5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6b4be616999a4213a63126e082cda27d'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:36:17.529054 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 952/1918] 10.222.0.30 () {58 vars in 1224 bytes} [Tue Sep 6 04:36:17 2022] DELETE /identity/v3/users/6b4be616999a4213a63126e082cda27d => generated 0 bytes in 514 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 04:36:17.562562 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:36:18.024932 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d8d2d8-8898-4ac5-8aeb-33f7f56cb908 service cinder] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:36:18.038763 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d8d2d8-8898-4ac5-8aeb-33f7f56cb908 service cinder] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:36:18.105241 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d6713f-ff56-4e33-87c2-f1ca612d8b23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:36:18.105930 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d6713f-ff56-4e33-87c2-f1ca612d8b23 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:36:18.106210 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d6713f-ff56-4e33-87c2-f1ca612d8b23 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:36:18.106483 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d6713f-ff56-4e33-87c2-f1ca612d8b23 admin admin] PATH_INFO: `/v3/projects/3a0efc68c5194d3f83abe71ef9b89031` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:36:18.107704 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d6713f-ff56-4e33-87c2-f1ca612d8b23 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3a0efc68c5194d3f83abe71ef9b89031)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:36:18.109643 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d6713f-ff56-4e33-87c2-f1ca612d8b23 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:36:18.231662 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-41d6713f-ff56-4e33-87c2-f1ca612d8b23 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3a0efc68c5194d3f83abe71ef9b89031', 'request_id': 'req-41d6713f-ff56-4e33-87c2-f1ca612d8b23'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:36:18.238791 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-41d6713f-ff56-4e33-87c2-f1ca612d8b23 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3a0efc68c5194d3f83abe71ef9b89031', 'request_id': 'req-41d6713f-ff56-4e33-87c2-f1ca612d8b23'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:36:18.258149 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:36:18.316665 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-41d6713f-ff56-4e33-87c2-f1ca612d8b23 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3a0efc68c5194d3f83abe71ef9b89031', 'request_id': 'req-41d6713f-ff56-4e33-87c2-f1ca612d8b23'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:36:18.336102 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-41d6713f-ff56-4e33-87c2-f1ca612d8b23 admin admin] The token cache is being invalidate because project 3a0efc68c5194d3f83abe71ef9b89031 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81698) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 04:36:18.342195 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-41d6713f-ff56-4e33-87c2-f1ca612d8b23 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:36:18.342195 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-41d6713f-ff56-4e33-87c2-f1ca612d8b23 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:36:18.342195 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 967/1919] 10.222.0.30 () {58 vars in 1233 bytes} [Tue Sep 6 04:36:18 2022] DELETE /identity/v3/projects/3a0efc68c5194d3f83abe71ef9b89031 => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 04:36:18.344440 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15e5a18-1868-4688-94e4-abe5b82dd0cb None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:36:18.344832 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15e5a18-1868-4688-94e4-abe5b82dd0cb None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:36:18.345211 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15e5a18-1868-4688-94e4-abe5b82dd0cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:36:18.371238 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-b15e5a18-1868-4688-94e4-abe5b82dd0cb None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:36:18.371238 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-b15e5a18-1868-4688-94e4-abe5b82dd0cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:36:18.436305 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 953/1920] 10.222.0.30 () {58 vars in 958 bytes} [Tue Sep 6 04:36:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 04:36:18.444392 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06eddeea-cfb4-4762-836e-a3432e77436b None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:36:18.444672 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06eddeea-cfb4-4762-836e-a3432e77436b None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:36:18.445047 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06eddeea-cfb4-4762-836e-a3432e77436b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:36:18.470397 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-06eddeea-cfb4-4762-836e-a3432e77436b None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:36:18.471648 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-06eddeea-cfb4-4762-836e-a3432e77436b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:36:18.479642 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 968/1921] 10.222.0.30 () {58 vars in 958 bytes} [Tue Sep 6 04:36:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 04:36:18.483933 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15e5a18-1868-4688-94e4-abe5b82dd0cb None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:36:18.494353 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15e5a18-1868-4688-94e4-abe5b82dd0cb None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:36:18.499489 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb05d4f-38cd-4f1c-bed5-5f486ab57315 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:36:18.500109 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb05d4f-38cd-4f1c-bed5-5f486ab57315 admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:36:18.500392 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb05d4f-38cd-4f1c-bed5-5f486ab57315 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:36:18.500666 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb05d4f-38cd-4f1c-bed5-5f486ab57315 admin admin] PATH_INFO: `/v3/domains` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:36:18.501338 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb05d4f-38cd-4f1c-bed5-5f486ab57315 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:36:18.501647 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb05d4f-38cd-4f1c-bed5-5f486ab57315 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:36:18.503296 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb05d4f-38cd-4f1c-bed5-5f486ab57315 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:36:18.516439 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 954/1922] 10.222.0.30 () {58 vars in 1153 bytes} [Tue Sep 6 04:36:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:36:18.520824 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06eddeea-cfb4-4762-836e-a3432e77436b None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:36:18.530794 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06eddeea-cfb4-4762-836e-a3432e77436b None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:36:18.535458 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11929f85-060b-4d71-873b-d558a73e546b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:36:18.536133 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11929f85-060b-4d71-873b-d558a73e546b admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:36:18.536414 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11929f85-060b-4d71-873b-d558a73e546b admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:36:18.536691 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11929f85-060b-4d71-873b-d558a73e546b admin admin] PATH_INFO: `/v3/projects` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:36:18.537453 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11929f85-060b-4d71-873b-d558a73e546b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:36:18.539392 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11929f85-060b-4d71-873b-d558a73e546b admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:36:18.643708 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 969/1923] 10.222.0.30 () {60 vars in 1153 bytes} [Tue Sep 6 04:36:18 2022] POST /identity/v3/projects => generated 373 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 04:36:18.648181 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb05d4f-38cd-4f1c-bed5-5f486ab57315 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:36:18.659067 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb05d4f-38cd-4f1c-bed5-5f486ab57315 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:36:18.716166 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ff4c42-e16c-445f-a33b-4997d6683dd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:36:18.716702 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ff4c42-e16c-445f-a33b-4997d6683dd4 admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:36:18.717049 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ff4c42-e16c-445f-a33b-4997d6683dd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:36:18.717247 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ff4c42-e16c-445f-a33b-4997d6683dd4 admin admin] PATH_INFO: `/v3/users` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:36:18.717943 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ff4c42-e16c-445f-a33b-4997d6683dd4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:36:18.719850 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ff4c42-e16c-445f-a33b-4997d6683dd4 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:36:18.825340 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 955/1924] 10.222.0.30 () {60 vars in 1144 bytes} [Tue Sep 6 04:36:18 2022] POST /identity/v3/users => generated 287 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 04:36:18.830047 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11929f85-060b-4d71-873b-d558a73e546b admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:36:18.841162 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11929f85-060b-4d71-873b-d558a73e546b admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:36:18.845832 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ac8087-b758-4b81-a9ac-afe0fa9c6673 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:36:18.846438 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ac8087-b758-4b81-a9ac-afe0fa9c6673 admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:36:18.846724 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ac8087-b758-4b81-a9ac-afe0fa9c6673 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:36:18.847027 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ac8087-b758-4b81-a9ac-afe0fa9c6673 admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:36:18.847687 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ac8087-b758-4b81-a9ac-afe0fa9c6673 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:36:18.849292 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ac8087-b758-4b81-a9ac-afe0fa9c6673 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:36:18.860425 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 970/1925] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Sep 6 04:36:18 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 06 04:36:18.864216 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ff4c42-e16c-445f-a33b-4997d6683dd4 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:36:18.870307 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:DELETE=11,keystone:SELECT=88,keystone:INSERT=1 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:36:18.874889 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ff4c42-e16c-445f-a33b-4997d6683dd4 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:36:18.879657 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08807e3-36c0-4b25-a187-3b49a33c10b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:36:18.880688 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08807e3-36c0-4b25-a187-3b49a33c10b4 admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:36:18.880688 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08807e3-36c0-4b25-a187-3b49a33c10b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:36:18.880863 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08807e3-36c0-4b25-a187-3b49a33c10b4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:36:18.881445 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c08807e3-36c0-4b25-a187-3b49a33c10b4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:36:18.883080 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c08807e3-36c0-4b25-a187-3b49a33c10b4 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:36:18.899078 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 956/1926] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Sep 6 04:36:18 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 04:36:18.900028 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ac8087-b758-4b81-a9ac-afe0fa9c6673 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:36:18.912197 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ac8087-b758-4b81-a9ac-afe0fa9c6673 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:36:18.920504 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48831c3a-3577-46e4-92c1-2b9ce3106a5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:36:18.921124 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48831c3a-3577-46e4-92c1-2b9ce3106a5d admin admin] REQUEST_METHOD: `PUT` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:36:18.921456 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48831c3a-3577-46e4-92c1-2b9ce3106a5d admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:36:18.921737 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48831c3a-3577-46e4-92c1-2b9ce3106a5d admin admin] PATH_INFO: `/v3/projects/59a926d403014fb1af2278bcefd5f5d8/users/d5188a098a124abda71016938eaec759/roles/29607b2de39f40d68bb6ceef64b8a732` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:36:18.923291 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:36:18.938478 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48831c3a-3577-46e4-92c1-2b9ce3106a5d admin admin] RBAC: Authorizing `identity:create_grant(project_id=59a926d403014fb1af2278bcefd5f5d8, user_id=d5188a098a124abda71016938eaec759, role_id=29607b2de39f40d68bb6ceef64b8a732)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:36:18.941133 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48831c3a-3577-46e4-92c1-2b9ce3106a5d admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:36:19.125424 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 971/1927] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Sep 6 04:36:18 2022] PUT /identity/v3/projects/59a926d403014fb1af2278bcefd5f5d8/users/d5188a098a124abda71016938eaec759/roles/29607b2de39f40d68bb6ceef64b8a732 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 04:36:19.128695 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25be8507-fdb4-44ea-b396-e692fe72148c None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:36:19.129242 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25be8507-fdb4-44ea-b396-e692fe72148c None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:36:19.129881 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25be8507-fdb4-44ea-b396-e692fe72148c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:36:19.174465 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-25be8507-fdb4-44ea-b396-e692fe72148c None None] MFA Rules not processed for user `d5188a098a124abda71016938eaec759`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:36:19.175846 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-25be8507-fdb4-44ea-b396-e692fe72148c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:36:19.251224 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 957/1928] 10.222.0.30 () {58 vars in 958 bytes} [Tue Sep 6 04:36:19 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 04:36:19.263566 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48831c3a-3577-46e4-92c1-2b9ce3106a5d admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:36:19.265442 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-48831c3a-3577-46e4-92c1-2b9ce3106a5d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:36:19.373344 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48831c3a-3577-46e4-92c1-2b9ce3106a5d admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:36:19.391450 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f35650-95bd-4bb4-bb05-acb679a95dc8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd09b5ab675634a2eb4b03755743f7d0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b707733ce65c42edbc1fd95ad0a1c06c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:36:19.391450 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f35650-95bd-4bb4-bb05-acb679a95dc8 service cinder] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:36:19.391450 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f35650-95bd-4bb4-bb05-acb679a95dc8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:36:19.391450 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f35650-95bd-4bb4-bb05-acb679a95dc8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:36:19.401710 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f35650-95bd-4bb4-bb05-acb679a95dc8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:36:19.403536 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f35650-95bd-4bb4-bb05-acb679a95dc8 service cinder] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:36:19.415579 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 972/1929] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Sep 6 04:36:19 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 06 04:36:19.439721 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25be8507-fdb4-44ea-b396-e692fe72148c None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:36:19.516238 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25be8507-fdb4-44ea-b396-e692fe72148c None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:36:19.521224 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02502d45-f66a-48a4-9324-af4a2edf0eb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:36:19.522053 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02502d45-f66a-48a4-9324-af4a2edf0eb5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:36:19.522340 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02502d45-f66a-48a4-9324-af4a2edf0eb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:36:19.522613 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02502d45-f66a-48a4-9324-af4a2edf0eb5 admin admin] PATH_INFO: `/v3/users/d5188a098a124abda71016938eaec759` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:36:19.523796 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02502d45-f66a-48a4-9324-af4a2edf0eb5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d5188a098a124abda71016938eaec759)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:36:19.526069 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02502d45-f66a-48a4-9324-af4a2edf0eb5 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:36:19.550796 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=89,keystone:DELETE=9,keystone:INSERT=2 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:36:19.654146 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:36:19.719375 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-02502d45-f66a-48a4-9324-af4a2edf0eb5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd5188a098a124abda71016938eaec759'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:36:19.806850 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-02502d45-f66a-48a4-9324-af4a2edf0eb5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd5188a098a124abda71016938eaec759'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:36:19.807479 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-02502d45-f66a-48a4-9324-af4a2edf0eb5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd5188a098a124abda71016938eaec759'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:36:19.815420 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-02502d45-f66a-48a4-9324-af4a2edf0eb5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd5188a098a124abda71016938eaec759'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:36:19.827964 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-02502d45-f66a-48a4-9324-af4a2edf0eb5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd5188a098a124abda71016938eaec759'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:36:19.834237 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 958/1930] 10.222.0.30 () {58 vars in 1224 bytes} [Tue Sep 6 04:36:19 2022] DELETE /identity/v3/users/d5188a098a124abda71016938eaec759 => generated 0 bytes in 395 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 04:36:19.838636 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1726363c-4c29-444c-8813-ea14f2d28bfe None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:36:19.845039 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1726363c-4c29-444c-8813-ea14f2d28bfe None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:36:19.845039 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1726363c-4c29-444c-8813-ea14f2d28bfe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:36:19.861752 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-1726363c-4c29-444c-8813-ea14f2d28bfe None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:36:19.871243 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-1726363c-4c29-444c-8813-ea14f2d28bfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:36:19.887335 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:36:19.923311 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 973/1931] 10.222.0.30 () {58 vars in 958 bytes} [Tue Sep 6 04:36:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 04:36:19.933873 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02502d45-f66a-48a4-9324-af4a2edf0eb5 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:36:19.936100 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-02502d45-f66a-48a4-9324-af4a2edf0eb5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:36:20.002164 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02502d45-f66a-48a4-9324-af4a2edf0eb5 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:36:20.005809 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7f355b-b023-47f9-a2ef-4668564d9a4f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90492440f6a6471e8392361c098428aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b707733ce65c42edbc1fd95ad0a1c06c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:36:20.008054 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7f355b-b023-47f9-a2ef-4668564d9a4f service neutron] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:36:20.008054 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7f355b-b023-47f9-a2ef-4668564d9a4f service neutron] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:36:20.008054 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7f355b-b023-47f9-a2ef-4668564d9a4f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:36:20.017443 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f7f355b-b023-47f9-a2ef-4668564d9a4f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:36:20.018884 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f7f355b-b023-47f9-a2ef-4668564d9a4f service neutron] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:36:20.023050 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:36:20.029754 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 959/1932] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Sep 6 04:36:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 04:36:20.217402 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:36:20.525413 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:36:20.644030 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1726363c-4c29-444c-8813-ea14f2d28bfe None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:36:20.651489 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-1726363c-4c29-444c-8813-ea14f2d28bfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:36:20.659391 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1726363c-4c29-444c-8813-ea14f2d28bfe None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:36:20.664388 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cedd7157-59b1-4929-b5cd-d8ef7e44c4d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:36:20.665233 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedd7157-59b1-4929-b5cd-d8ef7e44c4d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:36:20.665703 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedd7157-59b1-4929-b5cd-d8ef7e44c4d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:36:20.666131 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedd7157-59b1-4929-b5cd-d8ef7e44c4d3 admin admin] PATH_INFO: `/v3/projects/59a926d403014fb1af2278bcefd5f5d8` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:36:20.667946 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cedd7157-59b1-4929-b5cd-d8ef7e44c4d3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=59a926d403014fb1af2278bcefd5f5d8)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:36:20.670176 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cedd7157-59b1-4929-b5cd-d8ef7e44c4d3 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:36:20.806198 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-cedd7157-59b1-4929-b5cd-d8ef7e44c4d3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '59a926d403014fb1af2278bcefd5f5d8', 'request_id': 'req-cedd7157-59b1-4929-b5cd-d8ef7e44c4d3'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:36:20.812211 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-cedd7157-59b1-4929-b5cd-d8ef7e44c4d3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '59a926d403014fb1af2278bcefd5f5d8', 'request_id': 'req-cedd7157-59b1-4929-b5cd-d8ef7e44c4d3'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:36:20.836235 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:36:20.892929 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-cedd7157-59b1-4929-b5cd-d8ef7e44c4d3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '59a926d403014fb1af2278bcefd5f5d8', 'request_id': 'req-cedd7157-59b1-4929-b5cd-d8ef7e44c4d3'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:36:20.908002 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-cedd7157-59b1-4929-b5cd-d8ef7e44c4d3 admin admin] The token cache is being invalidate because project 59a926d403014fb1af2278bcefd5f5d8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81698) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 04:36:20.908417 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-cedd7157-59b1-4929-b5cd-d8ef7e44c4d3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:36:20.909162 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-cedd7157-59b1-4929-b5cd-d8ef7e44c4d3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:36:20.911135 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 974/1933] 10.222.0.30 () {58 vars in 1233 bytes} [Tue Sep 6 04:36:20 2022] DELETE /identity/v3/projects/59a926d403014fb1af2278bcefd5f5d8 => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 04:36:20.916336 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af34276a-81ef-421b-9c1a-f59038d0d16f None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:36:20.916980 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af34276a-81ef-421b-9c1a-f59038d0d16f None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:36:20.917596 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af34276a-81ef-421b-9c1a-f59038d0d16f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:36:20.938972 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-af34276a-81ef-421b-9c1a-f59038d0d16f None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:36:20.940403 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-af34276a-81ef-421b-9c1a-f59038d0d16f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:36:21.001813 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:36:21.008288 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 960/1934] 10.222.0.30 () {58 vars in 958 bytes} [Tue Sep 6 04:36:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 04:36:21.017362 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224517a8-8264-4d02-96a5-4d1b7e954ed6 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:36:21.017651 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224517a8-8264-4d02-96a5-4d1b7e954ed6 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:36:21.017925 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224517a8-8264-4d02-96a5-4d1b7e954ed6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:36:21.040363 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-224517a8-8264-4d02-96a5-4d1b7e954ed6 None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:36:21.041046 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-224517a8-8264-4d02-96a5-4d1b7e954ed6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:36:21.050072 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 975/1935] 10.222.0.30 () {58 vars in 958 bytes} [Tue Sep 6 04:36:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 04:36:21.058266 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af34276a-81ef-421b-9c1a-f59038d0d16f None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:36:21.068203 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af34276a-81ef-421b-9c1a-f59038d0d16f None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:36:21.072727 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e63a7a-8f97-4c71-af84-277301ac7e6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:36:21.073373 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e63a7a-8f97-4c71-af84-277301ac7e6f admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:36:21.074169 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e63a7a-8f97-4c71-af84-277301ac7e6f admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:36:21.074388 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e63a7a-8f97-4c71-af84-277301ac7e6f admin admin] PATH_INFO: `/v3/domains` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:36:21.075044 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e63a7a-8f97-4c71-af84-277301ac7e6f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:36:21.075381 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e63a7a-8f97-4c71-af84-277301ac7e6f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:36:21.076988 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e63a7a-8f97-4c71-af84-277301ac7e6f admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:36:21.088713 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 961/1936] 10.222.0.30 () {58 vars in 1153 bytes} [Tue Sep 6 04:36:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:36:21.093831 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224517a8-8264-4d02-96a5-4d1b7e954ed6 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:36:21.099333 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:36:21.103407 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224517a8-8264-4d02-96a5-4d1b7e954ed6 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:36:21.108862 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663c4f70-786f-4133-aa19-81b7754e41b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:36:21.109620 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663c4f70-786f-4133-aa19-81b7754e41b8 admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:36:21.109954 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663c4f70-786f-4133-aa19-81b7754e41b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:36:21.110240 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663c4f70-786f-4133-aa19-81b7754e41b8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:36:21.111027 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663c4f70-786f-4133-aa19-81b7754e41b8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:36:21.113078 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663c4f70-786f-4133-aa19-81b7754e41b8 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:36:21.132669 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:36:21.204406 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 976/1937] 10.222.0.30 () {60 vars in 1153 bytes} [Tue Sep 6 04:36:21 2022] POST /identity/v3/projects => generated 361 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 04:36:21.208281 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e63a7a-8f97-4c71-af84-277301ac7e6f admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:36:21.223810 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e63a7a-8f97-4c71-af84-277301ac7e6f admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:36:21.277213 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:36:21.302416 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cc6507-07f8-4b8e-a8b7-99aad995ade4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:36:21.304642 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cc6507-07f8-4b8e-a8b7-99aad995ade4 admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:36:21.304642 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cc6507-07f8-4b8e-a8b7-99aad995ade4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:36:21.304642 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cc6507-07f8-4b8e-a8b7-99aad995ade4 admin admin] PATH_INFO: `/v3/users` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:36:21.304642 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13cc6507-07f8-4b8e-a8b7-99aad995ade4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:36:21.306346 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13cc6507-07f8-4b8e-a8b7-99aad995ade4 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:36:21.372625 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:36:21.388463 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 962/1938] 10.222.0.30 () {60 vars in 1144 bytes} [Tue Sep 6 04:36:21 2022] POST /identity/v3/users => generated 281 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 04:36:21.389889 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663c4f70-786f-4133-aa19-81b7754e41b8 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:36:21.401162 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663c4f70-786f-4133-aa19-81b7754e41b8 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:36:21.406355 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd08509-5691-4271-b40d-6d5a00f73aa3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:36:21.406959 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd08509-5691-4271-b40d-6d5a00f73aa3 admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:36:21.407241 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd08509-5691-4271-b40d-6d5a00f73aa3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:36:21.407539 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd08509-5691-4271-b40d-6d5a00f73aa3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:36:21.408188 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd08509-5691-4271-b40d-6d5a00f73aa3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:36:21.409818 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd08509-5691-4271-b40d-6d5a00f73aa3 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:36:21.420796 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 977/1939] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Sep 6 04:36:21 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 06 04:36:21.424916 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cc6507-07f8-4b8e-a8b7-99aad995ade4 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:36:21.435800 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cc6507-07f8-4b8e-a8b7-99aad995ade4 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:36:21.442425 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d11d10a-d6b8-4642-986d-9732431eb9eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:36:21.443062 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d11d10a-d6b8-4642-986d-9732431eb9eb admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:36:21.443338 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d11d10a-d6b8-4642-986d-9732431eb9eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:36:21.443611 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d11d10a-d6b8-4642-986d-9732431eb9eb admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:36:21.444264 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d11d10a-d6b8-4642-986d-9732431eb9eb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:36:21.445941 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d11d10a-d6b8-4642-986d-9732431eb9eb admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:36:21.460135 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 963/1940] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Sep 6 04:36:21 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 04:36:21.465561 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd08509-5691-4271-b40d-6d5a00f73aa3 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:36:21.476199 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:36:21.477670 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd08509-5691-4271-b40d-6d5a00f73aa3 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:36:21.483697 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9227c9-0423-47b7-b8fc-80c977b9937e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:36:21.484373 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9227c9-0423-47b7-b8fc-80c977b9937e admin admin] REQUEST_METHOD: `PUT` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:36:21.484847 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9227c9-0423-47b7-b8fc-80c977b9937e admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:36:21.485127 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9227c9-0423-47b7-b8fc-80c977b9937e admin admin] PATH_INFO: `/v3/projects/adaddc97f25e491391fe5a34f9b7615e/users/234b2388cb8c43a1b558048925b2dc14/roles/29607b2de39f40d68bb6ceef64b8a732` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:36:21.501556 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:36:21.505919 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa9227c9-0423-47b7-b8fc-80c977b9937e admin admin] RBAC: Authorizing `identity:create_grant(project_id=adaddc97f25e491391fe5a34f9b7615e, user_id=234b2388cb8c43a1b558048925b2dc14, role_id=29607b2de39f40d68bb6ceef64b8a732)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:36:21.508601 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa9227c9-0423-47b7-b8fc-80c977b9937e admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:36:21.580130 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 978/1941] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Sep 6 04:36:21 2022] PUT /identity/v3/projects/adaddc97f25e491391fe5a34f9b7615e/users/234b2388cb8c43a1b558048925b2dc14/roles/29607b2de39f40d68bb6ceef64b8a732 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 04:36:21.591149 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80df9670-daeb-4576-9599-1e201fa89e14 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:36:21.591430 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80df9670-daeb-4576-9599-1e201fa89e14 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:36:21.591702 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80df9670-daeb-4576-9599-1e201fa89e14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:36:21.614422 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=98,keystone:INSERT=1,keystone:DELETE=1 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:36:21.649924 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-80df9670-daeb-4576-9599-1e201fa89e14 None None] MFA Rules not processed for user `234b2388cb8c43a1b558048925b2dc14`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:36:21.651771 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-80df9670-daeb-4576-9599-1e201fa89e14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:36:21.653420 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:36:21.739504 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 964/1942] 10.222.0.30 () {58 vars in 958 bytes} [Tue Sep 6 04:36:21 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 04:36:21.776348 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9227c9-0423-47b7-b8fc-80c977b9937e admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:36:21.778336 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-fa9227c9-0423-47b7-b8fc-80c977b9937e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:36:21.838657 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9227c9-0423-47b7-b8fc-80c977b9937e admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:36:21.843228 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03445b0-31bd-4e50-8907-2c99d296c53c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd09b5ab675634a2eb4b03755743f7d0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b707733ce65c42edbc1fd95ad0a1c06c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:36:21.843844 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03445b0-31bd-4e50-8907-2c99d296c53c service cinder] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:36:21.844129 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03445b0-31bd-4e50-8907-2c99d296c53c service cinder] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:36:21.844415 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03445b0-31bd-4e50-8907-2c99d296c53c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:36:21.854566 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03445b0-31bd-4e50-8907-2c99d296c53c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:36:21.857258 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03445b0-31bd-4e50-8907-2c99d296c53c service cinder] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:36:21.867166 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 979/1943] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Sep 6 04:36:21 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 04:36:21.882932 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80df9670-daeb-4576-9599-1e201fa89e14 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:36:21.946128 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80df9670-daeb-4576-9599-1e201fa89e14 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:36:21.950767 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bac09c2-4be1-4f6c-b27b-fb47468ca99e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:36:21.951403 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bac09c2-4be1-4f6c-b27b-fb47468ca99e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:36:21.951675 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bac09c2-4be1-4f6c-b27b-fb47468ca99e admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:36:21.951943 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bac09c2-4be1-4f6c-b27b-fb47468ca99e admin admin] PATH_INFO: `/v3/users/234b2388cb8c43a1b558048925b2dc14` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:36:21.953064 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bac09c2-4be1-4f6c-b27b-fb47468ca99e admin admin] RBAC: Authorizing `identity:delete_user(user_id=234b2388cb8c43a1b558048925b2dc14)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:36:21.954998 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bac09c2-4be1-4f6c-b27b-fb47468ca99e admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:36:22.182098 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-9bac09c2-4be1-4f6c-b27b-fb47468ca99e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '234b2388cb8c43a1b558048925b2dc14'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:36:22.195166 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:DELETE=9,keystone:SELECT=89,keystone:INSERT=2 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:36:22.257292 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-9bac09c2-4be1-4f6c-b27b-fb47468ca99e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '234b2388cb8c43a1b558048925b2dc14'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:36:22.257957 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-9bac09c2-4be1-4f6c-b27b-fb47468ca99e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '234b2388cb8c43a1b558048925b2dc14'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:36:22.266033 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-9bac09c2-4be1-4f6c-b27b-fb47468ca99e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '234b2388cb8c43a1b558048925b2dc14'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:36:22.278721 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-9bac09c2-4be1-4f6c-b27b-fb47468ca99e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '234b2388cb8c43a1b558048925b2dc14'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:36:22.284732 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 965/1944] 10.222.0.30 () {58 vars in 1224 bytes} [Tue Sep 6 04:36:21 2022] DELETE /identity/v3/users/234b2388cb8c43a1b558048925b2dc14 => generated 0 bytes in 402 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 04:36:22.290021 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fe8550-0250-47e4-ad19-5dabf49428d3 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:36:22.290368 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fe8550-0250-47e4-ad19-5dabf49428d3 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:36:22.290627 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fe8550-0250-47e4-ad19-5dabf49428d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:36:22.318256 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-f3fe8550-0250-47e4-ad19-5dabf49428d3 None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:36:22.319797 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-f3fe8550-0250-47e4-ad19-5dabf49428d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:36:22.356147 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:36:22.415512 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 980/1945] 10.222.0.30 () {58 vars in 958 bytes} [Tue Sep 6 04:36:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 04:36:22.425230 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bac09c2-4be1-4f6c-b27b-fb47468ca99e admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:36:22.426899 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-9bac09c2-4be1-4f6c-b27b-fb47468ca99e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:36:22.487583 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bac09c2-4be1-4f6c-b27b-fb47468ca99e admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:36:22.491324 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42303303-26ef-4bd5-8948-e656a11864a3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90492440f6a6471e8392361c098428aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b707733ce65c42edbc1fd95ad0a1c06c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:36:22.493992 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42303303-26ef-4bd5-8948-e656a11864a3 service neutron] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:36:22.493992 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42303303-26ef-4bd5-8948-e656a11864a3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:36:22.493992 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42303303-26ef-4bd5-8948-e656a11864a3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:36:22.502462 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42303303-26ef-4bd5-8948-e656a11864a3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:36:22.503989 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42303303-26ef-4bd5-8948-e656a11864a3 service neutron] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:36:22.515982 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 966/1946] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Sep 6 04:36:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 04:36:22.519202 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:DELETE=7,keystone:SELECT=88,keystone:INSERT=5 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:36:22.531917 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:36:22.927641 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:36:23.014739 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:36:23.020762 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fe8550-0250-47e4-ad19-5dabf49428d3 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:36:23.022248 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-f3fe8550-0250-47e4-ad19-5dabf49428d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:36:23.036812 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fe8550-0250-47e4-ad19-5dabf49428d3 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:36:23.041034 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7abd289d-4f4c-4017-a7db-9ffe508e5e1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:36:23.041744 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abd289d-4f4c-4017-a7db-9ffe508e5e1b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:36:23.042025 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abd289d-4f4c-4017-a7db-9ffe508e5e1b admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:36:23.042691 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abd289d-4f4c-4017-a7db-9ffe508e5e1b admin admin] PATH_INFO: `/v3/projects/adaddc97f25e491391fe5a34f9b7615e` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:36:23.043832 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7abd289d-4f4c-4017-a7db-9ffe508e5e1b admin admin] RBAC: Authorizing `identity:delete_project(project_id=adaddc97f25e491391fe5a34f9b7615e)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:36:23.045944 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7abd289d-4f4c-4017-a7db-9ffe508e5e1b admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:36:23.217027 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-7abd289d-4f4c-4017-a7db-9ffe508e5e1b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'adaddc97f25e491391fe5a34f9b7615e', 'request_id': 'req-7abd289d-4f4c-4017-a7db-9ffe508e5e1b'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:36:23.222694 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-7abd289d-4f4c-4017-a7db-9ffe508e5e1b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'adaddc97f25e491391fe5a34f9b7615e', 'request_id': 'req-7abd289d-4f4c-4017-a7db-9ffe508e5e1b'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:36:23.258219 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:36:23.296417 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-7abd289d-4f4c-4017-a7db-9ffe508e5e1b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'adaddc97f25e491391fe5a34f9b7615e', 'request_id': 'req-7abd289d-4f4c-4017-a7db-9ffe508e5e1b'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:36:23.313337 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-7abd289d-4f4c-4017-a7db-9ffe508e5e1b admin admin] The token cache is being invalidate because project adaddc97f25e491391fe5a34f9b7615e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81698) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 04:36:23.313735 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-7abd289d-4f4c-4017-a7db-9ffe508e5e1b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:36:23.314383 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-7abd289d-4f4c-4017-a7db-9ffe508e5e1b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:36:23.316135 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 981/1947] 10.222.0.30 () {58 vars in 1233 bytes} [Tue Sep 6 04:36:23 2022] DELETE /identity/v3/projects/adaddc97f25e491391fe5a34f9b7615e => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 04:36:23.321406 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee417261-8762-4ba2-8430-597bdcd0db77 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:36:23.322099 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee417261-8762-4ba2-8430-597bdcd0db77 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:36:23.324891 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee417261-8762-4ba2-8430-597bdcd0db77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:36:23.344282 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-ee417261-8762-4ba2-8430-597bdcd0db77 None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:36:23.345540 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-ee417261-8762-4ba2-8430-597bdcd0db77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:36:23.423941 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 967/1948] 10.222.0.30 () {58 vars in 958 bytes} [Tue Sep 6 04:36:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 04:36:23.431746 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa6e076-32c4-4ca9-8eb6-c6c9b163a379 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:36:23.432063 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa6e076-32c4-4ca9-8eb6-c6c9b163a379 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:36:23.432340 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa6e076-32c4-4ca9-8eb6-c6c9b163a379 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:36:23.465012 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-5fa6e076-32c4-4ca9-8eb6-c6c9b163a379 None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:36:23.466484 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-5fa6e076-32c4-4ca9-8eb6-c6c9b163a379 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:36:23.475104 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 982/1949] 10.222.0.30 () {58 vars in 958 bytes} [Tue Sep 6 04:36:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 04:36:23.483115 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee417261-8762-4ba2-8430-597bdcd0db77 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:36:23.492923 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee417261-8762-4ba2-8430-597bdcd0db77 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:36:23.498969 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ea3e06-4eb0-4c42-89c1-2edf1a243996 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:36:23.499597 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ea3e06-4eb0-4c42-89c1-2edf1a243996 admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:36:23.499878 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ea3e06-4eb0-4c42-89c1-2edf1a243996 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:36:23.500147 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ea3e06-4eb0-4c42-89c1-2edf1a243996 admin admin] PATH_INFO: `/v3/domains` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:36:23.500804 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ea3e06-4eb0-4c42-89c1-2edf1a243996 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:36:23.501108 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ea3e06-4eb0-4c42-89c1-2edf1a243996 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:36:23.502841 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ea3e06-4eb0-4c42-89c1-2edf1a243996 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:36:23.517342 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 968/1950] 10.222.0.30 () {58 vars in 1153 bytes} [Tue Sep 6 04:36:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:36:23.519956 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa6e076-32c4-4ca9-8eb6-c6c9b163a379 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:36:23.530267 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa6e076-32c4-4ca9-8eb6-c6c9b163a379 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:36:23.534822 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774dda1f-a86f-4c52-9a34-aa9c724adc65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:36:23.535444 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774dda1f-a86f-4c52-9a34-aa9c724adc65 admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:36:23.535693 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774dda1f-a86f-4c52-9a34-aa9c724adc65 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:36:23.535964 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774dda1f-a86f-4c52-9a34-aa9c724adc65 admin admin] PATH_INFO: `/v3/projects` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:36:23.536615 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-774dda1f-a86f-4c52-9a34-aa9c724adc65 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:36:23.539611 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-774dda1f-a86f-4c52-9a34-aa9c724adc65 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:36:23.688135 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:36:23.694448 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ea3e06-4eb0-4c42-89c1-2edf1a243996 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:36:23.707922 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ea3e06-4eb0-4c42-89c1-2edf1a243996 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:36:23.709193 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 983/1951] 10.222.0.30 () {60 vars in 1153 bytes} [Tue Sep 6 04:36:23 2022] POST /identity/v3/projects => generated 369 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 04:36:23.798152 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b09ac3-5b5b-440a-9999-3a2f3622cdea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:36:23.798799 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b09ac3-5b5b-440a-9999-3a2f3622cdea admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:36:23.799272 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b09ac3-5b5b-440a-9999-3a2f3622cdea admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:36:23.799556 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b09ac3-5b5b-440a-9999-3a2f3622cdea admin admin] PATH_INFO: `/v3/users` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:36:23.800331 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b09ac3-5b5b-440a-9999-3a2f3622cdea admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:36:23.804799 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b09ac3-5b5b-440a-9999-3a2f3622cdea admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:36:23.918761 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 969/1952] 10.222.0.30 () {60 vars in 1144 bytes} [Tue Sep 6 04:36:23 2022] POST /identity/v3/users => generated 285 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 04:36:23.919990 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774dda1f-a86f-4c52-9a34-aa9c724adc65 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:36:23.930802 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774dda1f-a86f-4c52-9a34-aa9c724adc65 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:36:23.935494 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59964b2-3bfc-4547-8534-af010376393c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:36:23.936284 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59964b2-3bfc-4547-8534-af010376393c admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:36:23.936564 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59964b2-3bfc-4547-8534-af010376393c admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:36:23.936836 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59964b2-3bfc-4547-8534-af010376393c admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:36:23.937532 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59964b2-3bfc-4547-8534-af010376393c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:36:23.939155 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59964b2-3bfc-4547-8534-af010376393c admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:36:23.954033 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 984/1953] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Sep 6 04:36:23 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 04:36:23.955308 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b09ac3-5b5b-440a-9999-3a2f3622cdea admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:36:23.967096 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b09ac3-5b5b-440a-9999-3a2f3622cdea admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:36:23.975762 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176b6351-d228-4a3d-8449-c838d4b85138 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:36:23.975762 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176b6351-d228-4a3d-8449-c838d4b85138 admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:36:23.975762 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176b6351-d228-4a3d-8449-c838d4b85138 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:36:23.975762 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176b6351-d228-4a3d-8449-c838d4b85138 admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:36:23.976133 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176b6351-d228-4a3d-8449-c838d4b85138 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:36:23.976133 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176b6351-d228-4a3d-8449-c838d4b85138 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:36:23.988391 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 970/1954] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Sep 6 04:36:23 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 04:36:23.992804 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59964b2-3bfc-4547-8534-af010376393c admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:36:24.003515 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59964b2-3bfc-4547-8534-af010376393c admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:36:24.008342 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ed26a0-743d-41e6-8348-8540cb60af8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:36:24.009139 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ed26a0-743d-41e6-8348-8540cb60af8c admin admin] REQUEST_METHOD: `PUT` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:36:24.009619 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ed26a0-743d-41e6-8348-8540cb60af8c admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:36:24.009996 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ed26a0-743d-41e6-8348-8540cb60af8c admin admin] PATH_INFO: `/v3/projects/45eb0ce3dada4e7db08a12ae882ca974/users/42246e779460402595923ebc2a3cb38a/roles/29607b2de39f40d68bb6ceef64b8a732` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:36:24.027451 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ed26a0-743d-41e6-8348-8540cb60af8c admin admin] RBAC: Authorizing `identity:create_grant(project_id=45eb0ce3dada4e7db08a12ae882ca974, user_id=42246e779460402595923ebc2a3cb38a, role_id=29607b2de39f40d68bb6ceef64b8a732)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:36:24.030424 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ed26a0-743d-41e6-8348-8540cb60af8c admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:36:24.102372 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 985/1955] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Sep 6 04:36:23 2022] PUT /identity/v3/projects/45eb0ce3dada4e7db08a12ae882ca974/users/42246e779460402595923ebc2a3cb38a/roles/29607b2de39f40d68bb6ceef64b8a732 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 04:36:24.114036 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a29a56-b0ec-4a0a-95e3-5a1c8acf27f8 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:36:24.114341 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a29a56-b0ec-4a0a-95e3-5a1c8acf27f8 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:36:24.114615 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a29a56-b0ec-4a0a-95e3-5a1c8acf27f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:36:24.171656 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-f6a29a56-b0ec-4a0a-95e3-5a1c8acf27f8 None None] MFA Rules not processed for user `42246e779460402595923ebc2a3cb38a`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:36:24.173891 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-f6a29a56-b0ec-4a0a-95e3-5a1c8acf27f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:36:24.229491 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=95,keystone:INSERT=2,keystone:DELETE=3 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:36:24.274134 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 971/1956] 10.222.0.30 () {58 vars in 958 bytes} [Tue Sep 6 04:36:24 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 04:36:24.286606 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ed26a0-743d-41e6-8348-8540cb60af8c admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:36:24.288873 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-94ed26a0-743d-41e6-8348-8540cb60af8c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:36:24.307481 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:36:24.418975 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ed26a0-743d-41e6-8348-8540cb60af8c admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:36:24.424478 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68f16c2-0b36-4b20-a30c-11794a43d2f2 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae5c31fb1d0342c0937bbc83a415113d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b707733ce65c42edbc1fd95ad0a1c06c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:36:24.425116 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68f16c2-0b36-4b20-a30c-11794a43d2f2 service glance] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:36:24.425699 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68f16c2-0b36-4b20-a30c-11794a43d2f2 service glance] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:36:24.426273 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68f16c2-0b36-4b20-a30c-11794a43d2f2 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:36:24.437727 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c68f16c2-0b36-4b20-a30c-11794a43d2f2 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:36:24.439366 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c68f16c2-0b36-4b20-a30c-11794a43d2f2 service glance] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:36:24.452379 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 986/1957] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Sep 6 04:36:24 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 04:36:25.322327 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:36:25.409451 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:36:26.171576 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a29a56-b0ec-4a0a-95e3-5a1c8acf27f8 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:36:26.173034 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-f6a29a56-b0ec-4a0a-95e3-5a1c8acf27f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:36:26.264915 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a29a56-b0ec-4a0a-95e3-5a1c8acf27f8 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:36:26.271385 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67b4995-4b53-45ab-81a8-69d83e2ec52c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:36:26.272112 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67b4995-4b53-45ab-81a8-69d83e2ec52c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:36:26.272479 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67b4995-4b53-45ab-81a8-69d83e2ec52c admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:36:26.273586 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67b4995-4b53-45ab-81a8-69d83e2ec52c admin admin] PATH_INFO: `/v3/users/c87bf43bae59424691dab57abc4a685b` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:36:26.275425 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67b4995-4b53-45ab-81a8-69d83e2ec52c admin admin] RBAC: Authorizing `identity:delete_user(user_id=c87bf43bae59424691dab57abc4a685b)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:36:26.278087 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67b4995-4b53-45ab-81a8-69d83e2ec52c admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:36:26.378960 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:36:26.498951 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=93,keystone:INSERT=2,keystone:DELETE=5 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:36:26.510288 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-d67b4995-4b53-45ab-81a8-69d83e2ec52c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c87bf43bae59424691dab57abc4a685b'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:36:26.596030 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-d67b4995-4b53-45ab-81a8-69d83e2ec52c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c87bf43bae59424691dab57abc4a685b'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:36:26.596792 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-d67b4995-4b53-45ab-81a8-69d83e2ec52c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c87bf43bae59424691dab57abc4a685b'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:36:26.607072 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-d67b4995-4b53-45ab-81a8-69d83e2ec52c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c87bf43bae59424691dab57abc4a685b'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:36:26.632375 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-d67b4995-4b53-45ab-81a8-69d83e2ec52c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c87bf43bae59424691dab57abc4a685b'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:36:26.658605 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 972/1958] 10.222.0.30 () {58 vars in 1224 bytes} [Tue Sep 6 04:36:26 2022] DELETE /identity/v3/users/c87bf43bae59424691dab57abc4a685b => generated 0 bytes in 483 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 04:36:26.658945 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f07abd-9186-47c7-b196-4bdda24596cc None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:36:26.659272 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f07abd-9186-47c7-b196-4bdda24596cc None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:36:26.659758 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f07abd-9186-47c7-b196-4bdda24596cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:36:26.699490 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-a0f07abd-9186-47c7-b196-4bdda24596cc None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:36:26.702025 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-a0f07abd-9186-47c7-b196-4bdda24596cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:36:26.849731 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 987/1959] 10.222.0.30 () {58 vars in 958 bytes} [Tue Sep 6 04:36:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 04:36:26.861835 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67b4995-4b53-45ab-81a8-69d83e2ec52c admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:36:26.863602 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-d67b4995-4b53-45ab-81a8-69d83e2ec52c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:36:26.907561 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:36:26.988946 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67b4995-4b53-45ab-81a8-69d83e2ec52c admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:36:26.988946 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274338b1-c714-4bc2-a0fd-f9c65cd4c767 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90492440f6a6471e8392361c098428aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b707733ce65c42edbc1fd95ad0a1c06c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:36:26.988946 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274338b1-c714-4bc2-a0fd-f9c65cd4c767 service neutron] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:36:26.988946 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274338b1-c714-4bc2-a0fd-f9c65cd4c767 service neutron] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:36:26.989349 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274338b1-c714-4bc2-a0fd-f9c65cd4c767 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:36:27.002328 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274338b1-c714-4bc2-a0fd-f9c65cd4c767 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:36:27.006028 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274338b1-c714-4bc2-a0fd-f9c65cd4c767 service neutron] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:36:27.037684 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 973/1960] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Sep 6 04:36:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 04:36:27.397876 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:36:27.592343 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f07abd-9186-47c7-b196-4bdda24596cc None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:36:27.593182 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-a0f07abd-9186-47c7-b196-4bdda24596cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:36:27.606230 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f07abd-9186-47c7-b196-4bdda24596cc None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:36:27.611134 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af15b53-495f-4940-b538-9fefffb0d537 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:36:27.623611 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af15b53-495f-4940-b538-9fefffb0d537 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:36:27.623912 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af15b53-495f-4940-b538-9fefffb0d537 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:36:27.624256 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af15b53-495f-4940-b538-9fefffb0d537 admin admin] PATH_INFO: `/v3/projects/706079ad6e194baf9bcdc5daee82a949` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:36:27.625778 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af15b53-495f-4940-b538-9fefffb0d537 admin admin] RBAC: Authorizing `identity:delete_project(project_id=706079ad6e194baf9bcdc5daee82a949)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:36:27.628577 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af15b53-495f-4940-b538-9fefffb0d537 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:36:27.676634 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274338b1-c714-4bc2-a0fd-f9c65cd4c767 service neutron] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:36:27.678590 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-274338b1-c714-4bc2-a0fd-f9c65cd4c767 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:36:27.693040 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274338b1-c714-4bc2-a0fd-f9c65cd4c767 service neutron] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:36:27.697828 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b5bb80-b23a-4240-96ef-3049e05f42b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:36:27.699073 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b5bb80-b23a-4240-96ef-3049e05f42b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:36:27.699465 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b5bb80-b23a-4240-96ef-3049e05f42b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:36:27.699749 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b5bb80-b23a-4240-96ef-3049e05f42b7 admin admin] PATH_INFO: `/v3/users/7ebc0166ad01461294d58ad5c5998633` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:36:27.701082 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b5bb80-b23a-4240-96ef-3049e05f42b7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7ebc0166ad01461294d58ad5c5998633)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:36:27.702956 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b5bb80-b23a-4240-96ef-3049e05f42b7 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:36:27.830100 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-6af15b53-495f-4940-b538-9fefffb0d537 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '706079ad6e194baf9bcdc5daee82a949', 'request_id': 'req-6af15b53-495f-4940-b538-9fefffb0d537'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:36:27.835681 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-6af15b53-495f-4940-b538-9fefffb0d537 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '706079ad6e194baf9bcdc5daee82a949', 'request_id': 'req-6af15b53-495f-4940-b538-9fefffb0d537'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:36:27.961424 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-6af15b53-495f-4940-b538-9fefffb0d537 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '706079ad6e194baf9bcdc5daee82a949', 'request_id': 'req-6af15b53-495f-4940-b538-9fefffb0d537'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:36:27.974375 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-6af15b53-495f-4940-b538-9fefffb0d537 admin admin] The token cache is being invalidate because project 706079ad6e194baf9bcdc5daee82a949 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81698) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 04:36:27.978883 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-6af15b53-495f-4940-b538-9fefffb0d537 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:36:27.978883 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-6af15b53-495f-4940-b538-9fefffb0d537 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:36:27.978883 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 988/1961] 10.222.0.30 () {58 vars in 1233 bytes} [Tue Sep 6 04:36:27 2022] DELETE /identity/v3/projects/706079ad6e194baf9bcdc5daee82a949 => generated 0 bytes in 386 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 04:36:27.982790 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af15b53-495f-4940-b538-9fefffb0d537 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:36:27.984550 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-6af15b53-495f-4940-b538-9fefffb0d537 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:36:27.984931 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-b5b5bb80-b23a-4240-96ef-3049e05f42b7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7ebc0166ad01461294d58ad5c5998633'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:36:28.050536 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af15b53-495f-4940-b538-9fefffb0d537 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:36:28.055314 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c73ff9-95e7-4519-977d-99be2cf6400f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:36:28.055897 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c73ff9-95e7-4519-977d-99be2cf6400f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:36:28.056179 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c73ff9-95e7-4519-977d-99be2cf6400f admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:36:28.056474 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c73ff9-95e7-4519-977d-99be2cf6400f admin admin] PATH_INFO: `/v3/users/6445823c10e342b0bea67f61383ab645` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:36:28.057662 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c73ff9-95e7-4519-977d-99be2cf6400f admin admin] RBAC: Authorizing `identity:delete_user(user_id=6445823c10e342b0bea67f61383ab645)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:36:28.064822 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-b5b5bb80-b23a-4240-96ef-3049e05f42b7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7ebc0166ad01461294d58ad5c5998633'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:36:28.064822 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-b5b5bb80-b23a-4240-96ef-3049e05f42b7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7ebc0166ad01461294d58ad5c5998633'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:36:28.065034 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c73ff9-95e7-4519-977d-99be2cf6400f admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:36:28.067424 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-b5b5bb80-b23a-4240-96ef-3049e05f42b7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7ebc0166ad01461294d58ad5c5998633'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:36:28.080148 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-b5b5bb80-b23a-4240-96ef-3049e05f42b7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7ebc0166ad01461294d58ad5c5998633'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:36:28.086593 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 974/1962] 10.222.0.30 () {58 vars in 1224 bytes} [Tue Sep 6 04:36:27 2022] DELETE /identity/v3/users/7ebc0166ad01461294d58ad5c5998633 => generated 0 bytes in 410 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 04:36:28.093497 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a7e07d-3f0e-4c59-bd0e-07698b7f4935 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:36:28.093497 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a7e07d-3f0e-4c59-bd0e-07698b7f4935 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:36:28.093497 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a7e07d-3f0e-4c59-bd0e-07698b7f4935 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:36:28.111075 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-62a7e07d-3f0e-4c59-bd0e-07698b7f4935 None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:36:28.116472 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-62a7e07d-3f0e-4c59-bd0e-07698b7f4935 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:36:28.169791 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:36:28.172280 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 975/1963] 10.222.0.30 () {58 vars in 958 bytes} [Tue Sep 6 04:36:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 04:36:28.181966 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a7e07d-3f0e-4c59-bd0e-07698b7f4935 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:36:28.183591 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-62a7e07d-3f0e-4c59-bd0e-07698b7f4935 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:36:28.239694 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a7e07d-3f0e-4c59-bd0e-07698b7f4935 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:36:28.243270 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72eb9f70-5e3d-4797-9cb8-1e3044b78ed9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90492440f6a6471e8392361c098428aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b707733ce65c42edbc1fd95ad0a1c06c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:36:28.243876 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72eb9f70-5e3d-4797-9cb8-1e3044b78ed9 service neutron] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:36:28.244151 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72eb9f70-5e3d-4797-9cb8-1e3044b78ed9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:36:28.244419 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72eb9f70-5e3d-4797-9cb8-1e3044b78ed9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:36:28.253451 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-77c73ff9-95e7-4519-977d-99be2cf6400f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6445823c10e342b0bea67f61383ab645'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:36:28.254110 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72eb9f70-5e3d-4797-9cb8-1e3044b78ed9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:36:28.255556 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72eb9f70-5e3d-4797-9cb8-1e3044b78ed9 service neutron] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:36:28.266259 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 976/1964] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Sep 6 04:36:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 04:36:28.337350 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-77c73ff9-95e7-4519-977d-99be2cf6400f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6445823c10e342b0bea67f61383ab645'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:36:28.337928 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-77c73ff9-95e7-4519-977d-99be2cf6400f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6445823c10e342b0bea67f61383ab645'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:36:28.350442 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-77c73ff9-95e7-4519-977d-99be2cf6400f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6445823c10e342b0bea67f61383ab645'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:36:28.358211 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-77c73ff9-95e7-4519-977d-99be2cf6400f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6445823c10e342b0bea67f61383ab645'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:36:28.364708 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 989/1965] 10.222.0.30 () {58 vars in 1224 bytes} [Tue Sep 6 04:36:27 2022] DELETE /identity/v3/users/6445823c10e342b0bea67f61383ab645 => generated 0 bytes in 382 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 04:36:28.457712 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:36:28.688030 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72eb9f70-5e3d-4797-9cb8-1e3044b78ed9 service neutron] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:36:28.690025 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-72eb9f70-5e3d-4797-9cb8-1e3044b78ed9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:36:28.703758 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72eb9f70-5e3d-4797-9cb8-1e3044b78ed9 service neutron] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:36:28.781246 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce306f69-a440-47c6-a31c-688b98b77a5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:36:28.781997 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce306f69-a440-47c6-a31c-688b98b77a5e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:36:28.782279 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce306f69-a440-47c6-a31c-688b98b77a5e admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:36:28.782555 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce306f69-a440-47c6-a31c-688b98b77a5e admin admin] PATH_INFO: `/v3/projects/52bb42a8f0ec457483a21c769a9f4561` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:36:28.783684 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce306f69-a440-47c6-a31c-688b98b77a5e admin admin] RBAC: Authorizing `identity:delete_project(project_id=52bb42a8f0ec457483a21c769a9f4561)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:36:28.785616 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce306f69-a440-47c6-a31c-688b98b77a5e admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:36:28.842920 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c73ff9-95e7-4519-977d-99be2cf6400f admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:36:28.844385 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-77c73ff9-95e7-4519-977d-99be2cf6400f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:36:28.853219 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:36:28.859330 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c73ff9-95e7-4519-977d-99be2cf6400f admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:36:28.868119 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16eb06f0-366f-474c-9355-cb7db4a4faef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:36:28.868119 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16eb06f0-366f-474c-9355-cb7db4a4faef admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:36:28.868119 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16eb06f0-366f-474c-9355-cb7db4a4faef admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:36:28.868119 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16eb06f0-366f-474c-9355-cb7db4a4faef admin admin] PATH_INFO: `/v3/projects/b9576495e4e447d7b953b03118af9614` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:36:28.868811 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16eb06f0-366f-474c-9355-cb7db4a4faef admin admin] RBAC: Authorizing `identity:delete_project(project_id=b9576495e4e447d7b953b03118af9614)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:36:28.868811 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16eb06f0-366f-474c-9355-cb7db4a4faef admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:36:28.910583 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-ce306f69-a440-47c6-a31c-688b98b77a5e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '52bb42a8f0ec457483a21c769a9f4561', 'request_id': 'req-ce306f69-a440-47c6-a31c-688b98b77a5e'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:36:28.922207 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=93,keystone:INSERT=3,keystone:DELETE=4 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:36:29.001408 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-16eb06f0-366f-474c-9355-cb7db4a4faef admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b9576495e4e447d7b953b03118af9614', 'request_id': 'req-16eb06f0-366f-474c-9355-cb7db4a4faef'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:36:29.007590 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-16eb06f0-366f-474c-9355-cb7db4a4faef admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b9576495e4e447d7b953b03118af9614', 'request_id': 'req-16eb06f0-366f-474c-9355-cb7db4a4faef'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:36:29.007734 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-ce306f69-a440-47c6-a31c-688b98b77a5e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '52bb42a8f0ec457483a21c769a9f4561', 'request_id': 'req-ce306f69-a440-47c6-a31c-688b98b77a5e'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:36:29.016734 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-ce306f69-a440-47c6-a31c-688b98b77a5e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '52bb42a8f0ec457483a21c769a9f4561', 'request_id': 'req-ce306f69-a440-47c6-a31c-688b98b77a5e'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:36:29.028599 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-ce306f69-a440-47c6-a31c-688b98b77a5e admin admin] The token cache is being invalidate because project 52bb42a8f0ec457483a21c769a9f4561 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81697) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 04:36:29.028897 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-ce306f69-a440-47c6-a31c-688b98b77a5e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:36:29.029585 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-ce306f69-a440-47c6-a31c-688b98b77a5e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:36:29.034123 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 977/1966] 10.222.0.30 () {58 vars in 1233 bytes} [Tue Sep 6 04:36:28 2022] DELETE /identity/v3/projects/52bb42a8f0ec457483a21c769a9f4561 => generated 0 bytes in 344 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 04:36:29.037761 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce306f69-a440-47c6-a31c-688b98b77a5e admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:36:29.043239 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-ce306f69-a440-47c6-a31c-688b98b77a5e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:36:29.098033 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-16eb06f0-366f-474c-9355-cb7db4a4faef admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b9576495e4e447d7b953b03118af9614', 'request_id': 'req-16eb06f0-366f-474c-9355-cb7db4a4faef'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:36:29.107784 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce306f69-a440-47c6-a31c-688b98b77a5e admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:36:29.111170 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-16eb06f0-366f-474c-9355-cb7db4a4faef admin admin] The token cache is being invalidate because project b9576495e4e447d7b953b03118af9614 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81698) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 04:36:29.111495 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-16eb06f0-366f-474c-9355-cb7db4a4faef admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:36:29.112164 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-16eb06f0-366f-474c-9355-cb7db4a4faef admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:36:29.114151 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 990/1967] 10.222.0.30 () {58 vars in 1233 bytes} [Tue Sep 6 04:36:28 2022] DELETE /identity/v3/projects/b9576495e4e447d7b953b03118af9614 => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 04:36:29.118774 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632feed2-363c-4b7b-bfb8-a530024a0e7a None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:36:29.118998 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632feed2-363c-4b7b-bfb8-a530024a0e7a None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:36:29.119272 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632feed2-363c-4b7b-bfb8-a530024a0e7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:36:29.149750 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-632feed2-363c-4b7b-bfb8-a530024a0e7a None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:36:29.150946 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-632feed2-363c-4b7b-bfb8-a530024a0e7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:36:29.171138 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a2d26d-91a6-4855-a0eb-9de1b7e9abef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:36:29.171813 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a2d26d-91a6-4855-a0eb-9de1b7e9abef admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:36:29.172102 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a2d26d-91a6-4855-a0eb-9de1b7e9abef admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:36:29.172377 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a2d26d-91a6-4855-a0eb-9de1b7e9abef admin admin] PATH_INFO: `/v3/users/088224d4ba6c4dc599f1c184d534f089` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:36:29.173550 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a2d26d-91a6-4855-a0eb-9de1b7e9abef admin admin] RBAC: Authorizing `identity:delete_user(user_id=088224d4ba6c4dc599f1c184d534f089)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:36:29.175544 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a2d26d-91a6-4855-a0eb-9de1b7e9abef admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:36:29.216407 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 991/1968] 10.222.0.30 () {58 vars in 958 bytes} [Tue Sep 6 04:36:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 04:36:29.224634 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cd536a-80be-4362-805c-737c2caeddc3 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:36:29.226995 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cd536a-80be-4362-805c-737c2caeddc3 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:36:29.226995 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cd536a-80be-4362-805c-737c2caeddc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:36:29.245641 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-52cd536a-80be-4362-805c-737c2caeddc3 None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:36:29.246869 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-52cd536a-80be-4362-805c-737c2caeddc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:36:29.254909 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 992/1969] 10.222.0.30 () {58 vars in 958 bytes} [Tue Sep 6 04:36:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 04:36:29.259085 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52cd536a-80be-4362-805c-737c2caeddc3 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:36:29.269030 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52cd536a-80be-4362-805c-737c2caeddc3 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:36:29.273676 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5692c31-858c-457e-ac64-922c50c25159 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:36:29.274269 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5692c31-858c-457e-ac64-922c50c25159 admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:36:29.274818 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5692c31-858c-457e-ac64-922c50c25159 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:36:29.275081 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5692c31-858c-457e-ac64-922c50c25159 admin admin] PATH_INFO: `/v3/domains` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:36:29.275719 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5692c31-858c-457e-ac64-922c50c25159 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:36:29.276169 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5692c31-858c-457e-ac64-922c50c25159 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:36:29.278439 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5692c31-858c-457e-ac64-922c50c25159 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:36:29.301607 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 993/1970] 10.222.0.30 () {58 vars in 1153 bytes} [Tue Sep 6 04:36:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 04:36:29.304174 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5692c31-858c-457e-ac64-922c50c25159 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:36:29.314751 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5692c31-858c-457e-ac64-922c50c25159 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:36:29.319829 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6105ed-8335-4135-a93e-68bc71de785a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:36:29.320402 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6105ed-8335-4135-a93e-68bc71de785a admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:36:29.320679 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6105ed-8335-4135-a93e-68bc71de785a admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:36:29.320948 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6105ed-8335-4135-a93e-68bc71de785a admin admin] PATH_INFO: `/v3/projects` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:36:29.321604 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6105ed-8335-4135-a93e-68bc71de785a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:36:29.324166 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6105ed-8335-4135-a93e-68bc71de785a admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:36:29.393530 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-d7a2d26d-91a6-4855-a0eb-9de1b7e9abef admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '088224d4ba6c4dc599f1c184d534f089'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:36:29.415236 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 994/1971] 10.222.0.30 () {60 vars in 1153 bytes} [Tue Sep 6 04:36:29 2022] POST /identity/v3/projects => generated 387 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 04:36:29.419567 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6105ed-8335-4135-a93e-68bc71de785a admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:36:29.478931 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6105ed-8335-4135-a93e-68bc71de785a admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:36:29.483657 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366aa949-e4b9-44f4-b507-4b2a70d99a75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:36:29.484262 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366aa949-e4b9-44f4-b507-4b2a70d99a75 admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:36:29.484541 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366aa949-e4b9-44f4-b507-4b2a70d99a75 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:36:29.484818 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366aa949-e4b9-44f4-b507-4b2a70d99a75 admin admin] PATH_INFO: `/v3/users` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:36:29.485520 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366aa949-e4b9-44f4-b507-4b2a70d99a75 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:36:29.486721 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-d7a2d26d-91a6-4855-a0eb-9de1b7e9abef admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '088224d4ba6c4dc599f1c184d534f089'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:36:29.487409 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-d7a2d26d-91a6-4855-a0eb-9de1b7e9abef admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '088224d4ba6c4dc599f1c184d534f089'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:36:29.487510 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366aa949-e4b9-44f4-b507-4b2a70d99a75 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:36:29.495819 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-d7a2d26d-91a6-4855-a0eb-9de1b7e9abef admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '088224d4ba6c4dc599f1c184d534f089'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:36:29.509118 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-d7a2d26d-91a6-4855-a0eb-9de1b7e9abef admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '088224d4ba6c4dc599f1c184d534f089'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:36:29.515792 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 978/1972] 10.222.0.30 () {58 vars in 1224 bytes} [Tue Sep 6 04:36:29 2022] DELETE /identity/v3/users/088224d4ba6c4dc599f1c184d534f089 => generated 0 bytes in 478 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 04:36:29.550280 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:36:29.591270 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:36:29.593391 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 995/1973] 10.222.0.30 () {60 vars in 1144 bytes} [Tue Sep 6 04:36:29 2022] POST /identity/v3/users => generated 294 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 04:36:29.594555 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a2d26d-91a6-4855-a0eb-9de1b7e9abef admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:36:29.596003 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-d7a2d26d-91a6-4855-a0eb-9de1b7e9abef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:36:29.661116 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a2d26d-91a6-4855-a0eb-9de1b7e9abef admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:36:29.665781 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6950e28-eb2d-4b2f-91e8-676c5ab3699a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:36:29.666398 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6950e28-eb2d-4b2f-91e8-676c5ab3699a admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:36:29.666682 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6950e28-eb2d-4b2f-91e8-676c5ab3699a admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:36:29.666959 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6950e28-eb2d-4b2f-91e8-676c5ab3699a admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:36:29.668380 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6950e28-eb2d-4b2f-91e8-676c5ab3699a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:36:29.670026 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6950e28-eb2d-4b2f-91e8-676c5ab3699a admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:36:29.679807 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 979/1974] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Sep 6 04:36:29 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 06 04:36:29.683851 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366aa949-e4b9-44f4-b507-4b2a70d99a75 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:36:29.693973 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366aa949-e4b9-44f4-b507-4b2a70d99a75 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:36:29.699110 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99513a3e-d839-41ba-9f5e-a99b2847044b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:36:29.699713 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99513a3e-d839-41ba-9f5e-a99b2847044b admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:36:29.699991 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99513a3e-d839-41ba-9f5e-a99b2847044b admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:36:29.700265 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99513a3e-d839-41ba-9f5e-a99b2847044b admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:36:29.700905 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99513a3e-d839-41ba-9f5e-a99b2847044b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:36:29.702544 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99513a3e-d839-41ba-9f5e-a99b2847044b admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:36:29.712587 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 996/1975] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Sep 6 04:36:29 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 06 04:36:29.718103 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6950e28-eb2d-4b2f-91e8-676c5ab3699a admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:36:29.728352 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6950e28-eb2d-4b2f-91e8-676c5ab3699a admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:36:29.733906 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584d5f9c-6593-479f-a6a4-699e957b0a1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:36:29.734980 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584d5f9c-6593-479f-a6a4-699e957b0a1d admin admin] REQUEST_METHOD: `PUT` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:36:29.735315 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584d5f9c-6593-479f-a6a4-699e957b0a1d admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:36:29.735590 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584d5f9c-6593-479f-a6a4-699e957b0a1d admin admin] PATH_INFO: `/v3/projects/b0d6ad8924a84c15864b9677827d491c/users/41f31a5ac1b2439bacdaabd42cc33440/roles/29607b2de39f40d68bb6ceef64b8a732` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:36:29.751226 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-584d5f9c-6593-479f-a6a4-699e957b0a1d admin admin] RBAC: Authorizing `identity:create_grant(project_id=b0d6ad8924a84c15864b9677827d491c, user_id=41f31a5ac1b2439bacdaabd42cc33440, role_id=29607b2de39f40d68bb6ceef64b8a732)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:36:29.754146 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-584d5f9c-6593-479f-a6a4-699e957b0a1d admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:36:29.894322 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 980/1976] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Sep 6 04:36:29 2022] PUT /identity/v3/projects/b0d6ad8924a84c15864b9677827d491c/users/41f31a5ac1b2439bacdaabd42cc33440/roles/29607b2de39f40d68bb6ceef64b8a732 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 04:36:29.907429 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99513a3e-d839-41ba-9f5e-a99b2847044b admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:36:29.909697 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-99513a3e-d839-41ba-9f5e-a99b2847044b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:36:29.910907 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99abc20-6ae0-47ff-a1aa-f1912de1ce35 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:36:29.911268 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99abc20-6ae0-47ff-a1aa-f1912de1ce35 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:36:29.911482 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99abc20-6ae0-47ff-a1aa-f1912de1ce35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:36:29.942119 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-a99abc20-6ae0-47ff-a1aa-f1912de1ce35 None None] MFA Rules not processed for user `41f31a5ac1b2439bacdaabd42cc33440`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:36:29.943568 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-a99abc20-6ae0-47ff-a1aa-f1912de1ce35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:36:29.976335 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99513a3e-d839-41ba-9f5e-a99b2847044b admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:36:30.042845 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 981/1977] 10.222.0.30 () {58 vars in 958 bytes} [Tue Sep 6 04:36:29 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 04:36:30.046479 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb13438-c3a0-40dd-a942-17eca16dabf3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd09b5ab675634a2eb4b03755743f7d0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b707733ce65c42edbc1fd95ad0a1c06c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:36:30.046479 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb13438-c3a0-40dd-a942-17eca16dabf3 service cinder] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:36:30.046698 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99abc20-6ae0-47ff-a1aa-f1912de1ce35 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:36:30.046803 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb13438-c3a0-40dd-a942-17eca16dabf3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:36:30.046803 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb13438-c3a0-40dd-a942-17eca16dabf3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:36:30.047301 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-3eb13438-c3a0-40dd-a942-17eca16dabf3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:36:30.110720 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99abc20-6ae0-47ff-a1aa-f1912de1ce35 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:36:30.116675 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:36:30.117709 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9039b0c2-8955-4909-8378-7473965253bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:36:30.118595 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9039b0c2-8955-4909-8378-7473965253bc admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:36:30.118947 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9039b0c2-8955-4909-8378-7473965253bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:36:30.119372 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9039b0c2-8955-4909-8378-7473965253bc admin admin] PATH_INFO: `/v3/projects` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:36:30.120003 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9039b0c2-8955-4909-8378-7473965253bc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:36:30.122810 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9039b0c2-8955-4909-8378-7473965253bc admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:36:30.139372 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb13438-c3a0-40dd-a942-17eca16dabf3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:36:30.140827 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb13438-c3a0-40dd-a942-17eca16dabf3 service cinder] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:36:30.151123 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 997/1978] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Sep 6 04:36:29 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 244 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 04:36:30.159152 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb13438-c3a0-40dd-a942-17eca16dabf3 service cinder] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:36:30.161622 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-3eb13438-c3a0-40dd-a942-17eca16dabf3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:36:30.190416 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb13438-c3a0-40dd-a942-17eca16dabf3 service cinder] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:36:30.196812 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6d0a28-4679-4150-bffa-3c6a57e23d86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:36:30.197501 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6d0a28-4679-4150-bffa-3c6a57e23d86 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:36:30.197785 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6d0a28-4679-4150-bffa-3c6a57e23d86 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:36:30.198643 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6d0a28-4679-4150-bffa-3c6a57e23d86 admin admin] PATH_INFO: `/v3/projects/f4d8325f73d44d6db02cc9e937e3f218` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:36:30.201803 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6d0a28-4679-4150-bffa-3c6a57e23d86 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f4d8325f73d44d6db02cc9e937e3f218)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:36:30.203887 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6d0a28-4679-4150-bffa-3c6a57e23d86 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:36:30.220918 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 982/1979] 10.222.0.30 () {60 vars in 1153 bytes} [Tue Sep 6 04:36:30 2022] POST /identity/v3/projects => generated 385 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 04:36:30.224713 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9039b0c2-8955-4909-8378-7473965253bc admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:36:30.248948 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=96,keystone:INSERT=3,keystone:DELETE=1 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:36:30.278515 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:DELETE=12,keystone:SELECT=87,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:36:30.304304 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9039b0c2-8955-4909-8378-7473965253bc admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:36:30.311158 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65880609-09d6-4417-910c-81a00c4d1755 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:36:30.312104 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65880609-09d6-4417-910c-81a00c4d1755 admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:36:30.312370 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65880609-09d6-4417-910c-81a00c4d1755 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:36:30.312644 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65880609-09d6-4417-910c-81a00c4d1755 admin admin] PATH_INFO: `/v3/users` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:36:30.314123 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65880609-09d6-4417-910c-81a00c4d1755 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:36:30.317830 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65880609-09d6-4417-910c-81a00c4d1755 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:36:30.330347 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-5a6d0a28-4679-4150-bffa-3c6a57e23d86 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f4d8325f73d44d6db02cc9e937e3f218', 'request_id': 'req-5a6d0a28-4679-4150-bffa-3c6a57e23d86'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:36:30.336033 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-5a6d0a28-4679-4150-bffa-3c6a57e23d86 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f4d8325f73d44d6db02cc9e937e3f218', 'request_id': 'req-5a6d0a28-4679-4150-bffa-3c6a57e23d86'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:36:30.410561 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-5a6d0a28-4679-4150-bffa-3c6a57e23d86 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f4d8325f73d44d6db02cc9e937e3f218', 'request_id': 'req-5a6d0a28-4679-4150-bffa-3c6a57e23d86'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:36:30.415802 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 983/1980] 10.222.0.30 () {60 vars in 1144 bytes} [Tue Sep 6 04:36:30 2022] POST /identity/v3/users => generated 299 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 04:36:30.416910 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65880609-09d6-4417-910c-81a00c4d1755 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:36:30.425501 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-5a6d0a28-4679-4150-bffa-3c6a57e23d86 admin admin] The token cache is being invalidate because project f4d8325f73d44d6db02cc9e937e3f218 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81698) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 04:36:30.426074 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-5a6d0a28-4679-4150-bffa-3c6a57e23d86 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:36:30.426753 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-5a6d0a28-4679-4150-bffa-3c6a57e23d86 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:36:30.428341 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 998/1981] 10.222.0.30 () {58 vars in 1233 bytes} [Tue Sep 6 04:36:30 2022] DELETE /identity/v3/projects/f4d8325f73d44d6db02cc9e937e3f218 => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 04:36:30.429172 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65880609-09d6-4417-910c-81a00c4d1755 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:36:30.433051 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c337d65-3d6a-45a3-8438-a992cd5db8dc None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:36:30.433815 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c337d65-3d6a-45a3-8438-a992cd5db8dc None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:36:30.434187 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c337d65-3d6a-45a3-8438-a992cd5db8dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:36:30.455804 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-5c337d65-3d6a-45a3-8438-a992cd5db8dc None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:36:30.457048 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-5c337d65-3d6a-45a3-8438-a992cd5db8dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:36:30.503599 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce6c5f0-94ff-410d-b79e-f270414bea60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:36:30.504439 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce6c5f0-94ff-410d-b79e-f270414bea60 admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:36:30.504809 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce6c5f0-94ff-410d-b79e-f270414bea60 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:36:30.505198 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce6c5f0-94ff-410d-b79e-f270414bea60 admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:36:30.506073 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce6c5f0-94ff-410d-b79e-f270414bea60 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:36:30.508141 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce6c5f0-94ff-410d-b79e-f270414bea60 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:36:30.519477 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 984/1982] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Sep 6 04:36:30 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 04:36:30.521518 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 999/1983] 10.222.0.30 () {58 vars in 958 bytes} [Tue Sep 6 04:36:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 04:36:30.523934 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce6c5f0-94ff-410d-b79e-f270414bea60 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:36:30.526013 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-3ce6c5f0-94ff-410d-b79e-f270414bea60 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:36:30.530193 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85aefd1-46a4-4262-9a53-0910d63fbb1a None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:36:30.530545 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85aefd1-46a4-4262-9a53-0910d63fbb1a None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:36:30.530822 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85aefd1-46a4-4262-9a53-0910d63fbb1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:36:30.540557 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce6c5f0-94ff-410d-b79e-f270414bea60 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:36:30.545693 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66890bd7-3516-49b7-a230-0b4bdf4d6235 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:36:30.549786 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66890bd7-3516-49b7-a230-0b4bdf4d6235 admin admin] REQUEST_METHOD: `PUT` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:36:30.549786 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66890bd7-3516-49b7-a230-0b4bdf4d6235 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:36:30.549786 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66890bd7-3516-49b7-a230-0b4bdf4d6235 admin admin] PATH_INFO: `/v3/projects/8cd1cf5b505242328d0af4615cf58bf4/users/c99cfea1ce9d4a5c8b556933a06e9ace/roles/ea4c814713b149108e488de855b025e4` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:36:30.550471 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-f85aefd1-46a4-4262-9a53-0910d63fbb1a None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:36:30.551672 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-f85aefd1-46a4-4262-9a53-0910d63fbb1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:36:30.559736 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1000/1984] 10.222.0.30 () {58 vars in 958 bytes} [Tue Sep 6 04:36:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 04:36:30.563021 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66890bd7-3516-49b7-a230-0b4bdf4d6235 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8cd1cf5b505242328d0af4615cf58bf4, user_id=c99cfea1ce9d4a5c8b556933a06e9ace, role_id=ea4c814713b149108e488de855b025e4)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:36:30.564105 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85aefd1-46a4-4262-9a53-0910d63fbb1a None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:36:30.566311 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66890bd7-3516-49b7-a230-0b4bdf4d6235 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:36:30.574751 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85aefd1-46a4-4262-9a53-0910d63fbb1a None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:36:30.579772 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400e50fa-56aa-4638-bdad-a53baa04c0d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:36:30.580456 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400e50fa-56aa-4638-bdad-a53baa04c0d5 admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:36:30.580813 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400e50fa-56aa-4638-bdad-a53baa04c0d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:36:30.581093 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400e50fa-56aa-4638-bdad-a53baa04c0d5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:36:30.581777 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400e50fa-56aa-4638-bdad-a53baa04c0d5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:36:30.582083 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400e50fa-56aa-4638-bdad-a53baa04c0d5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:36:30.583715 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400e50fa-56aa-4638-bdad-a53baa04c0d5 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:36:30.595513 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1001/1985] 10.222.0.30 () {58 vars in 1153 bytes} [Tue Sep 6 04:36:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 04:36:30.599784 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400e50fa-56aa-4638-bdad-a53baa04c0d5 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:36:30.610112 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400e50fa-56aa-4638-bdad-a53baa04c0d5 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:36:30.614826 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725baff1-f620-46f9-b4fd-59c76a886bb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:36:30.615419 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725baff1-f620-46f9-b4fd-59c76a886bb9 admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:36:30.615701 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725baff1-f620-46f9-b4fd-59c76a886bb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:36:30.615979 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725baff1-f620-46f9-b4fd-59c76a886bb9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:36:30.616699 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-725baff1-f620-46f9-b4fd-59c76a886bb9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:36:30.618576 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-725baff1-f620-46f9-b4fd-59c76a886bb9 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:36:30.630514 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 985/1986] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Sep 6 04:36:30 2022] PUT /identity/v3/projects/8cd1cf5b505242328d0af4615cf58bf4/users/c99cfea1ce9d4a5c8b556933a06e9ace/roles/ea4c814713b149108e488de855b025e4 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 04:36:30.636221 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66890bd7-3516-49b7-a230-0b4bdf4d6235 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:36:30.702577 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66890bd7-3516-49b7-a230-0b4bdf4d6235 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:36:30.707131 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529396b4-627a-43a4-89fb-93d77392e075 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:36:30.707740 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529396b4-627a-43a4-89fb-93d77392e075 admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:36:30.708018 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529396b4-627a-43a4-89fb-93d77392e075 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:36:30.708293 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529396b4-627a-43a4-89fb-93d77392e075 admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:36:30.708933 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529396b4-627a-43a4-89fb-93d77392e075 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:36:30.710589 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529396b4-627a-43a4-89fb-93d77392e075 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:36:30.721028 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 986/1987] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Sep 6 04:36:30 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 06 04:36:30.725208 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1002/1988] 10.222.0.30 () {60 vars in 1153 bytes} [Tue Sep 6 04:36:30 2022] POST /identity/v3/projects => generated 375 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 04:36:30.725347 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529396b4-627a-43a4-89fb-93d77392e075 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:36:30.727461 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725baff1-f620-46f9-b4fd-59c76a886bb9 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:36:30.737099 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529396b4-627a-43a4-89fb-93d77392e075 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:36:30.799236 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725baff1-f620-46f9-b4fd-59c76a886bb9 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:36:30.801405 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfcd2a44-2b77-4a16-b44b-bb8651950a5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:36:30.802492 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcd2a44-2b77-4a16-b44b-bb8651950a5d admin admin] REQUEST_METHOD: `PUT` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:36:30.802778 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcd2a44-2b77-4a16-b44b-bb8651950a5d admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:36:30.803217 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcd2a44-2b77-4a16-b44b-bb8651950a5d admin admin] PATH_INFO: `/v3/projects/8cd1cf5b505242328d0af4615cf58bf4/users/c99cfea1ce9d4a5c8b556933a06e9ace/roles/ea4c814713b149108e488de855b025e4` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:36:30.805578 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806a81d3-5a6f-4f99-bb1e-c1544b867ed6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:36:30.805709 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfcd2a44-2b77-4a16-b44b-bb8651950a5d admin admin] RBAC: Authorizing `identity:create_grant(project_id=8cd1cf5b505242328d0af4615cf58bf4, user_id=c99cfea1ce9d4a5c8b556933a06e9ace, role_id=ea4c814713b149108e488de855b025e4)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:36:30.806366 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806a81d3-5a6f-4f99-bb1e-c1544b867ed6 admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:36:30.806643 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806a81d3-5a6f-4f99-bb1e-c1544b867ed6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:36:30.807021 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806a81d3-5a6f-4f99-bb1e-c1544b867ed6 admin admin] PATH_INFO: `/v3/users` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:36:30.808703 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-806a81d3-5a6f-4f99-bb1e-c1544b867ed6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:36:30.808982 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfcd2a44-2b77-4a16-b44b-bb8651950a5d admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:36:30.811061 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-806a81d3-5a6f-4f99-bb1e-c1544b867ed6 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:36:30.821200 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 987/1989] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Sep 6 04:36:30 2022] PUT /identity/v3/projects/8cd1cf5b505242328d0af4615cf58bf4/users/c99cfea1ce9d4a5c8b556933a06e9ace/roles/ea4c814713b149108e488de855b025e4 => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 04:36:30.837797 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4463944-7df2-4d41-9763-0e52492537b0 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:36:30.838086 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4463944-7df2-4d41-9763-0e52492537b0 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:36:30.838367 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4463944-7df2-4d41-9763-0e52492537b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:36:30.851421 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:36:30.872559 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-d4463944-7df2-4d41-9763-0e52492537b0 None None] MFA Rules not processed for user `c99cfea1ce9d4a5c8b556933a06e9ace`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:36:30.873867 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-d4463944-7df2-4d41-9763-0e52492537b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:36:30.895068 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=89,keystone:DELETE=6,keystone:INSERT=5 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:36:30.901028 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1003/1990] 10.222.0.30 () {60 vars in 1144 bytes} [Tue Sep 6 04:36:30 2022] POST /identity/v3/users => generated 288 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 04:36:30.905902 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806a81d3-5a6f-4f99-bb1e-c1544b867ed6 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:36:30.916154 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806a81d3-5a6f-4f99-bb1e-c1544b867ed6 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:36:30.951753 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 988/1991] 10.222.0.30 () {58 vars in 958 bytes} [Tue Sep 6 04:36:30 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 04:36:30.963073 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4463944-7df2-4d41-9763-0e52492537b0 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:36:30.965158 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-d4463944-7df2-4d41-9763-0e52492537b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:36:30.977384 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec49628-a4eb-4e93-8ade-f808121c38d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:36:30.983726 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec49628-a4eb-4e93-8ade-f808121c38d3 admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:36:30.983726 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec49628-a4eb-4e93-8ade-f808121c38d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:36:30.983726 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec49628-a4eb-4e93-8ade-f808121c38d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:36:30.983726 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec49628-a4eb-4e93-8ade-f808121c38d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:36:30.983726 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec49628-a4eb-4e93-8ade-f808121c38d3 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:36:30.992714 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1004/1992] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Sep 6 04:36:30 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 06 04:36:30.997695 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec49628-a4eb-4e93-8ade-f808121c38d3 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:36:31.010429 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec49628-a4eb-4e93-8ade-f808121c38d3 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:36:31.016235 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eadb3f1-61b2-4232-b975-777c2cb1488c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:36:31.017064 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eadb3f1-61b2-4232-b975-777c2cb1488c admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:36:31.017377 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eadb3f1-61b2-4232-b975-777c2cb1488c admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:36:31.017658 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eadb3f1-61b2-4232-b975-777c2cb1488c admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:36:31.018348 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eadb3f1-61b2-4232-b975-777c2cb1488c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:36:31.020095 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eadb3f1-61b2-4232-b975-777c2cb1488c admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:36:31.024611 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4463944-7df2-4d41-9763-0e52492537b0 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:36:31.031166 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d31bc59-0e39-4fc8-bc0c-240efd95cbd1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd09b5ab675634a2eb4b03755743f7d0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b707733ce65c42edbc1fd95ad0a1c06c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:36:31.031780 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d31bc59-0e39-4fc8-bc0c-240efd95cbd1 service cinder] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:36:31.032068 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d31bc59-0e39-4fc8-bc0c-240efd95cbd1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:36:31.032336 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d31bc59-0e39-4fc8-bc0c-240efd95cbd1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:36:31.032585 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1005/1993] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Sep 6 04:36:30 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 06 04:36:31.036918 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eadb3f1-61b2-4232-b975-777c2cb1488c admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:36:31.041763 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:36:31.044451 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d31bc59-0e39-4fc8-bc0c-240efd95cbd1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:36:31.045935 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d31bc59-0e39-4fc8-bc0c-240efd95cbd1 service cinder] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:36:31.047925 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eadb3f1-61b2-4232-b975-777c2cb1488c admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:36:31.052775 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4eb951-c4a1-4631-8083-8e7ad354435a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:36:31.053794 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4eb951-c4a1-4631-8083-8e7ad354435a admin admin] REQUEST_METHOD: `PUT` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:36:31.054088 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4eb951-c4a1-4631-8083-8e7ad354435a admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:36:31.054367 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4eb951-c4a1-4631-8083-8e7ad354435a admin admin] PATH_INFO: `/v3/projects/206a2b9767ec4997904d37fd41624f25/users/f43e14473d3848b58fcca06a52977eb8/roles/29607b2de39f40d68bb6ceef64b8a732` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:36:31.055620 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 989/1994] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Sep 6 04:36:30 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 06 04:36:31.070040 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4eb951-c4a1-4631-8083-8e7ad354435a admin admin] RBAC: Authorizing `identity:create_grant(project_id=206a2b9767ec4997904d37fd41624f25, user_id=f43e14473d3848b58fcca06a52977eb8, role_id=29607b2de39f40d68bb6ceef64b8a732)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:36:31.072762 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4eb951-c4a1-4631-8083-8e7ad354435a admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:36:31.137547 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1006/1995] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Sep 6 04:36:31 2022] PUT /identity/v3/projects/206a2b9767ec4997904d37fd41624f25/users/f43e14473d3848b58fcca06a52977eb8/roles/29607b2de39f40d68bb6ceef64b8a732 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 04:36:31.151986 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7683d7-e1c6-4505-bfca-ed9d6954781f None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:36:31.152322 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7683d7-e1c6-4505-bfca-ed9d6954781f None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:36:31.152597 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7683d7-e1c6-4505-bfca-ed9d6954781f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:36:31.183774 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-2c7683d7-e1c6-4505-bfca-ed9d6954781f None None] MFA Rules not processed for user `f43e14473d3848b58fcca06a52977eb8`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:36:31.184740 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-2c7683d7-e1c6-4505-bfca-ed9d6954781f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:36:31.254699 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 990/1996] 10.222.0.30 () {58 vars in 958 bytes} [Tue Sep 6 04:36:31 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 04:36:31.259458 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:36:31.267055 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4eb951-c4a1-4631-8083-8e7ad354435a admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:36:31.270417 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-0d4eb951-c4a1-4631-8083-8e7ad354435a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:36:31.328519 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4eb951-c4a1-4631-8083-8e7ad354435a admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:36:31.332216 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3b5c83-ebcd-40c4-a358-92018855de38 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd09b5ab675634a2eb4b03755743f7d0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b707733ce65c42edbc1fd95ad0a1c06c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:36:31.332855 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3b5c83-ebcd-40c4-a358-92018855de38 service cinder] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:36:31.333137 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3b5c83-ebcd-40c4-a358-92018855de38 service cinder] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:36:31.333440 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3b5c83-ebcd-40c4-a358-92018855de38 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:36:31.343694 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3b5c83-ebcd-40c4-a358-92018855de38 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:36:31.345220 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3b5c83-ebcd-40c4-a358-92018855de38 service cinder] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:36:31.355758 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1007/1997] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Sep 6 04:36:31 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 04:36:32.381142 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7683d7-e1c6-4505-bfca-ed9d6954781f None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:36:32.443718 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7683d7-e1c6-4505-bfca-ed9d6954781f None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:36:32.448223 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89c1261-b349-4bb4-8019-2197de0d121e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:36:32.449614 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89c1261-b349-4bb4-8019-2197de0d121e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:36:32.449928 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89c1261-b349-4bb4-8019-2197de0d121e admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:36:32.450200 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89c1261-b349-4bb4-8019-2197de0d121e admin admin] PATH_INFO: `/v3/users/41f31a5ac1b2439bacdaabd42cc33440` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:36:32.451331 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89c1261-b349-4bb4-8019-2197de0d121e admin admin] RBAC: Authorizing `identity:delete_user(user_id=41f31a5ac1b2439bacdaabd42cc33440)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:36:32.453233 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89c1261-b349-4bb4-8019-2197de0d121e admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:36:32.661723 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-d89c1261-b349-4bb4-8019-2197de0d121e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '41f31a5ac1b2439bacdaabd42cc33440'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:36:32.761784 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-d89c1261-b349-4bb4-8019-2197de0d121e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '41f31a5ac1b2439bacdaabd42cc33440'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:36:32.762417 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-d89c1261-b349-4bb4-8019-2197de0d121e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '41f31a5ac1b2439bacdaabd42cc33440'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:36:32.770001 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-d89c1261-b349-4bb4-8019-2197de0d121e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '41f31a5ac1b2439bacdaabd42cc33440'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:36:32.782884 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-d89c1261-b349-4bb4-8019-2197de0d121e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '41f31a5ac1b2439bacdaabd42cc33440'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:36:32.788699 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 991/1998] 10.222.0.30 () {58 vars in 1224 bytes} [Tue Sep 6 04:36:32 2022] DELETE /identity/v3/users/41f31a5ac1b2439bacdaabd42cc33440 => generated 0 bytes in 408 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 04:36:32.793200 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec668cf7-fc5f-4813-8909-cd65abaca3bf None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:36:32.793728 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec668cf7-fc5f-4813-8909-cd65abaca3bf None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:36:32.793971 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec668cf7-fc5f-4813-8909-cd65abaca3bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:36:32.817003 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-ec668cf7-fc5f-4813-8909-cd65abaca3bf None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:36:32.818259 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-ec668cf7-fc5f-4813-8909-cd65abaca3bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:36:32.885281 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1008/1999] 10.222.0.30 () {58 vars in 958 bytes} [Tue Sep 6 04:36:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 04:36:32.895816 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89c1261-b349-4bb4-8019-2197de0d121e admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:36:32.897291 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-d89c1261-b349-4bb4-8019-2197de0d121e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:36:32.933469 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:36:32.957760 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89c1261-b349-4bb4-8019-2197de0d121e admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:36:32.961468 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa16596-e777-4eae-8c11-82792f7cbd32 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90492440f6a6471e8392361c098428aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b707733ce65c42edbc1fd95ad0a1c06c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:36:32.962092 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa16596-e777-4eae-8c11-82792f7cbd32 service neutron] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:36:32.962367 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa16596-e777-4eae-8c11-82792f7cbd32 service neutron] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:36:32.963262 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa16596-e777-4eae-8c11-82792f7cbd32 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:36:32.977834 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa16596-e777-4eae-8c11-82792f7cbd32 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:36:32.979439 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa16596-e777-4eae-8c11-82792f7cbd32 service neutron] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:36:32.991820 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 992/2000] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Sep 6 04:36:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 04:36:33.448806 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec668cf7-fc5f-4813-8909-cd65abaca3bf None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:36:33.453099 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-ec668cf7-fc5f-4813-8909-cd65abaca3bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:36:33.458556 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:36:33.462559 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec668cf7-fc5f-4813-8909-cd65abaca3bf None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:36:33.469042 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d09b5d-d13f-4870-8559-26bf503514cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:36:33.469894 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d09b5d-d13f-4870-8559-26bf503514cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:36:33.483762 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d09b5d-d13f-4870-8559-26bf503514cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:36:33.483762 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d09b5d-d13f-4870-8559-26bf503514cb admin admin] PATH_INFO: `/v3/projects/b0d6ad8924a84c15864b9677827d491c` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:36:33.483762 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d09b5d-d13f-4870-8559-26bf503514cb admin admin] RBAC: Authorizing `identity:delete_project(project_id=b0d6ad8924a84c15864b9677827d491c)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:36:33.483762 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d09b5d-d13f-4870-8559-26bf503514cb admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:36:33.606512 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-63d09b5d-d13f-4870-8559-26bf503514cb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b0d6ad8924a84c15864b9677827d491c', 'request_id': 'req-63d09b5d-d13f-4870-8559-26bf503514cb'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:36:33.611902 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-63d09b5d-d13f-4870-8559-26bf503514cb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b0d6ad8924a84c15864b9677827d491c', 'request_id': 'req-63d09b5d-d13f-4870-8559-26bf503514cb'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:36:33.710001 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-63d09b5d-d13f-4870-8559-26bf503514cb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b0d6ad8924a84c15864b9677827d491c', 'request_id': 'req-63d09b5d-d13f-4870-8559-26bf503514cb'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:36:33.722400 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-63d09b5d-d13f-4870-8559-26bf503514cb admin admin] The token cache is being invalidate because project b0d6ad8924a84c15864b9677827d491c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81698) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 04:36:33.722691 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-63d09b5d-d13f-4870-8559-26bf503514cb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:36:33.723331 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-63d09b5d-d13f-4870-8559-26bf503514cb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:36:33.724823 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1009/2001] 10.222.0.30 () {58 vars in 1233 bytes} [Tue Sep 6 04:36:33 2022] DELETE /identity/v3/projects/b0d6ad8924a84c15864b9677827d491c => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 04:36:33.728789 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa16596-e777-4eae-8c11-82792f7cbd32 service neutron] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:36:33.738675 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa16596-e777-4eae-8c11-82792f7cbd32 service neutron] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:36:33.802004 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c375d67a-6257-46fc-9d0b-79eddf0f1a3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:36:33.802748 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c375d67a-6257-46fc-9d0b-79eddf0f1a3b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:36:33.803030 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c375d67a-6257-46fc-9d0b-79eddf0f1a3b admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:36:33.803309 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c375d67a-6257-46fc-9d0b-79eddf0f1a3b admin admin] PATH_INFO: `/v3/users/c99cfea1ce9d4a5c8b556933a06e9ace` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:36:33.804634 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c375d67a-6257-46fc-9d0b-79eddf0f1a3b admin admin] RBAC: Authorizing `identity:delete_user(user_id=c99cfea1ce9d4a5c8b556933a06e9ace)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:36:33.806857 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c375d67a-6257-46fc-9d0b-79eddf0f1a3b admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:36:33.970457 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-c375d67a-6257-46fc-9d0b-79eddf0f1a3b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c99cfea1ce9d4a5c8b556933a06e9ace'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:36:34.042156 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-c375d67a-6257-46fc-9d0b-79eddf0f1a3b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c99cfea1ce9d4a5c8b556933a06e9ace'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:36:34.043333 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-c375d67a-6257-46fc-9d0b-79eddf0f1a3b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c99cfea1ce9d4a5c8b556933a06e9ace'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:36:34.051129 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-c375d67a-6257-46fc-9d0b-79eddf0f1a3b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c99cfea1ce9d4a5c8b556933a06e9ace'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:36:34.065034 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-c375d67a-6257-46fc-9d0b-79eddf0f1a3b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c99cfea1ce9d4a5c8b556933a06e9ace'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:36:34.072681 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 993/2002] 10.222.0.30 () {58 vars in 1224 bytes} [Tue Sep 6 04:36:33 2022] DELETE /identity/v3/users/c99cfea1ce9d4a5c8b556933a06e9ace => generated 0 bytes in 344 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 04:36:34.501866 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d09b5d-d13f-4870-8559-26bf503514cb admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:36:34.504948 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-63d09b5d-d13f-4870-8559-26bf503514cb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:36:34.544153 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d09b5d-d13f-4870-8559-26bf503514cb admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:36:34.633132 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4964757-0985-4665-b063-818ae48a8fdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:36:34.633812 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4964757-0985-4665-b063-818ae48a8fdc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:36:34.634109 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4964757-0985-4665-b063-818ae48a8fdc admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:36:34.634381 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4964757-0985-4665-b063-818ae48a8fdc admin admin] PATH_INFO: `/v3/projects/8cd1cf5b505242328d0af4615cf58bf4` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:36:34.635536 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4964757-0985-4665-b063-818ae48a8fdc admin admin] RBAC: Authorizing `identity:delete_project(project_id=8cd1cf5b505242328d0af4615cf58bf4)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:36:34.638064 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4964757-0985-4665-b063-818ae48a8fdc admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:36:34.827146 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-a4964757-0985-4665-b063-818ae48a8fdc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8cd1cf5b505242328d0af4615cf58bf4', 'request_id': 'req-a4964757-0985-4665-b063-818ae48a8fdc'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:36:34.834067 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-a4964757-0985-4665-b063-818ae48a8fdc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8cd1cf5b505242328d0af4615cf58bf4', 'request_id': 'req-a4964757-0985-4665-b063-818ae48a8fdc'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:36:34.952816 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-a4964757-0985-4665-b063-818ae48a8fdc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8cd1cf5b505242328d0af4615cf58bf4', 'request_id': 'req-a4964757-0985-4665-b063-818ae48a8fdc'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:36:34.981948 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-a4964757-0985-4665-b063-818ae48a8fdc admin admin] The token cache is being invalidate because project 8cd1cf5b505242328d0af4615cf58bf4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81698) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 04:36:34.982441 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-a4964757-0985-4665-b063-818ae48a8fdc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:36:34.983380 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-a4964757-0985-4665-b063-818ae48a8fdc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:36:34.987216 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1010/2003] 10.222.0.30 () {58 vars in 1233 bytes} [Tue Sep 6 04:36:34 2022] DELETE /identity/v3/projects/8cd1cf5b505242328d0af4615cf58bf4 => generated 0 bytes in 486 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 04:36:34.992602 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b8dec5-c13c-4e74-b9da-8f7631462347 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:36:34.994050 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b8dec5-c13c-4e74-b9da-8f7631462347 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:36:34.994050 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b8dec5-c13c-4e74-b9da-8f7631462347 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:36:35.028157 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-81b8dec5-c13c-4e74-b9da-8f7631462347 None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:36:35.029989 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-81b8dec5-c13c-4e74-b9da-8f7631462347 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:36:35.087891 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:36:35.102546 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 994/2004] 10.222.0.30 () {58 vars in 958 bytes} [Tue Sep 6 04:36:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 04:36:35.111096 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8c9a26-28fa-4a7e-82b7-03e41eab2d6d None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:36:35.111382 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8c9a26-28fa-4a7e-82b7-03e41eab2d6d None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:36:35.111656 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8c9a26-28fa-4a7e-82b7-03e41eab2d6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:36:35.138563 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-9c8c9a26-28fa-4a7e-82b7-03e41eab2d6d None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:36:35.140247 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-9c8c9a26-28fa-4a7e-82b7-03e41eab2d6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:36:35.163339 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1011/2005] 10.222.0.30 () {58 vars in 958 bytes} [Tue Sep 6 04:36:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 04:36:35.167741 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b8dec5-c13c-4e74-b9da-8f7631462347 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:36:35.177952 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b8dec5-c13c-4e74-b9da-8f7631462347 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:36:35.182644 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3de23f4-dc62-4500-b266-313597edc27d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:36:35.183248 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3de23f4-dc62-4500-b266-313597edc27d admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:36:35.183526 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3de23f4-dc62-4500-b266-313597edc27d admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:36:35.183795 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3de23f4-dc62-4500-b266-313597edc27d admin admin] PATH_INFO: `/v3/domains` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:36:35.185037 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3de23f4-dc62-4500-b266-313597edc27d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:36:35.185370 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3de23f4-dc62-4500-b266-313597edc27d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:36:35.187001 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3de23f4-dc62-4500-b266-313597edc27d admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:36:35.198383 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 995/2006] 10.222.0.30 () {58 vars in 1153 bytes} [Tue Sep 6 04:36:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:36:35.202922 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8c9a26-28fa-4a7e-82b7-03e41eab2d6d None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:36:35.211995 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8c9a26-28fa-4a7e-82b7-03e41eab2d6d None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:36:35.216737 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd939e2-eb73-4b95-8f4f-71a5bc64778f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:36:35.217374 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd939e2-eb73-4b95-8f4f-71a5bc64778f admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:36:35.217654 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd939e2-eb73-4b95-8f4f-71a5bc64778f admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:36:35.217924 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd939e2-eb73-4b95-8f4f-71a5bc64778f admin admin] PATH_INFO: `/v3/projects` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:36:35.218567 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd939e2-eb73-4b95-8f4f-71a5bc64778f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:36:35.220466 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd939e2-eb73-4b95-8f4f-71a5bc64778f admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:36:35.236739 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:36:35.334225 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1012/2007] 10.222.0.30 () {60 vars in 1153 bytes} [Tue Sep 6 04:36:35 2022] POST /identity/v3/projects => generated 385 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 04:36:35.349245 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3de23f4-dc62-4500-b266-313597edc27d admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:36:35.352989 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3de23f4-dc62-4500-b266-313597edc27d admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:36:35.429338 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1faecbcc-8735-44d4-8aa1-b2de7a1e1fdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:36:35.429978 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1faecbcc-8735-44d4-8aa1-b2de7a1e1fdb admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:36:35.430263 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1faecbcc-8735-44d4-8aa1-b2de7a1e1fdb admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:36:35.430538 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1faecbcc-8735-44d4-8aa1-b2de7a1e1fdb admin admin] PATH_INFO: `/v3/users` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:36:35.431179 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1faecbcc-8735-44d4-8aa1-b2de7a1e1fdb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:36:35.433146 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1faecbcc-8735-44d4-8aa1-b2de7a1e1fdb admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:36:35.538436 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 996/2008] 10.222.0.30 () {60 vars in 1144 bytes} [Tue Sep 6 04:36:35 2022] POST /identity/v3/users => generated 293 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 04:36:35.542706 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd939e2-eb73-4b95-8f4f-71a5bc64778f admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:36:35.553798 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd939e2-eb73-4b95-8f4f-71a5bc64778f admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:36:35.558335 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32aa3289-e0cf-4c61-8650-0afdfa4e9ad8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:36:35.558998 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32aa3289-e0cf-4c61-8650-0afdfa4e9ad8 admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:36:35.559241 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32aa3289-e0cf-4c61-8650-0afdfa4e9ad8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:36:35.559683 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32aa3289-e0cf-4c61-8650-0afdfa4e9ad8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:36:35.560446 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32aa3289-e0cf-4c61-8650-0afdfa4e9ad8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:36:35.562091 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32aa3289-e0cf-4c61-8650-0afdfa4e9ad8 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:36:35.571917 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1013/2009] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Sep 6 04:36:35 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 04:36:35.575980 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1faecbcc-8735-44d4-8aa1-b2de7a1e1fdb admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:36:35.586215 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1faecbcc-8735-44d4-8aa1-b2de7a1e1fdb admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:36:35.591745 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3946d0-bac1-48d2-a7c5-04b9160b9bb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:36:35.592351 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3946d0-bac1-48d2-a7c5-04b9160b9bb5 admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:36:35.592634 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3946d0-bac1-48d2-a7c5-04b9160b9bb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:36:35.592915 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3946d0-bac1-48d2-a7c5-04b9160b9bb5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:36:35.593588 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3946d0-bac1-48d2-a7c5-04b9160b9bb5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:36:35.595188 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3946d0-bac1-48d2-a7c5-04b9160b9bb5 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:36:35.615369 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 997/2010] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Sep 6 04:36:35 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 04:36:35.619651 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32aa3289-e0cf-4c61-8650-0afdfa4e9ad8 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:36:35.635645 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32aa3289-e0cf-4c61-8650-0afdfa4e9ad8 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:36:35.642405 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc121a5c-90de-4587-b9b4-d4d79ba6fc3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:36:35.643092 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc121a5c-90de-4587-b9b4-d4d79ba6fc3f admin admin] REQUEST_METHOD: `PUT` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:36:35.643374 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc121a5c-90de-4587-b9b4-d4d79ba6fc3f admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:36:35.643799 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc121a5c-90de-4587-b9b4-d4d79ba6fc3f admin admin] PATH_INFO: `/v3/projects/ea9efab8451a40919ea4206cc0a506da/users/ed694a74b8e540768ee5458fcb5ce649/roles/29607b2de39f40d68bb6ceef64b8a732` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:36:35.660328 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc121a5c-90de-4587-b9b4-d4d79ba6fc3f admin admin] RBAC: Authorizing `identity:create_grant(project_id=ea9efab8451a40919ea4206cc0a506da, user_id=ed694a74b8e540768ee5458fcb5ce649, role_id=29607b2de39f40d68bb6ceef64b8a732)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:36:35.663036 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc121a5c-90de-4587-b9b4-d4d79ba6fc3f admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:36:35.772865 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1014/2011] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Sep 6 04:36:35 2022] PUT /identity/v3/projects/ea9efab8451a40919ea4206cc0a506da/users/ed694a74b8e540768ee5458fcb5ce649/roles/29607b2de39f40d68bb6ceef64b8a732 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 04:36:35.785195 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7af39dd-047a-4fda-adde-3a2f8f566b5b None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:36:35.785612 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7af39dd-047a-4fda-adde-3a2f8f566b5b None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:36:35.785917 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7af39dd-047a-4fda-adde-3a2f8f566b5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:36:35.813946 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-b7af39dd-047a-4fda-adde-3a2f8f566b5b None None] MFA Rules not processed for user `ed694a74b8e540768ee5458fcb5ce649`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:36:35.815701 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-b7af39dd-047a-4fda-adde-3a2f8f566b5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:36:35.880193 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 998/2012] 10.222.0.30 () {58 vars in 958 bytes} [Tue Sep 6 04:36:35 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 04:36:35.892008 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc121a5c-90de-4587-b9b4-d4d79ba6fc3f admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:36:35.893791 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-fc121a5c-90de-4587-b9b4-d4d79ba6fc3f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:36:35.949774 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc121a5c-90de-4587-b9b4-d4d79ba6fc3f admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:36:35.953779 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d908449b-cf11-4b9c-8252-47f9007aec11 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd09b5ab675634a2eb4b03755743f7d0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b707733ce65c42edbc1fd95ad0a1c06c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:36:35.954662 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d908449b-cf11-4b9c-8252-47f9007aec11 service cinder] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:36:35.955079 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d908449b-cf11-4b9c-8252-47f9007aec11 service cinder] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:36:35.955485 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d908449b-cf11-4b9c-8252-47f9007aec11 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:36:35.967797 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d908449b-cf11-4b9c-8252-47f9007aec11 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:36:35.970059 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d908449b-cf11-4b9c-8252-47f9007aec11 service cinder] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:36:35.983810 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1015/2013] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Sep 6 04:36:35 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 04:36:36.770412 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7af39dd-047a-4fda-adde-3a2f8f566b5b None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:36:36.771909 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-b7af39dd-047a-4fda-adde-3a2f8f566b5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:36:36.823277 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:36:36.838839 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7af39dd-047a-4fda-adde-3a2f8f566b5b None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:36:36.843361 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38f6ff3-b24d-42d9-93dc-aaa92a461c54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:36:36.851143 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38f6ff3-b24d-42d9-93dc-aaa92a461c54 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:36:36.851143 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38f6ff3-b24d-42d9-93dc-aaa92a461c54 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:36:36.851143 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38f6ff3-b24d-42d9-93dc-aaa92a461c54 admin admin] PATH_INFO: `/v3/users/f43e14473d3848b58fcca06a52977eb8` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:36:36.851143 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b38f6ff3-b24d-42d9-93dc-aaa92a461c54 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f43e14473d3848b58fcca06a52977eb8)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:36:36.851143 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b38f6ff3-b24d-42d9-93dc-aaa92a461c54 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:36:37.076295 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-b38f6ff3-b24d-42d9-93dc-aaa92a461c54 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f43e14473d3848b58fcca06a52977eb8'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:36:37.153987 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-b38f6ff3-b24d-42d9-93dc-aaa92a461c54 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f43e14473d3848b58fcca06a52977eb8'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:36:37.154603 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-b38f6ff3-b24d-42d9-93dc-aaa92a461c54 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f43e14473d3848b58fcca06a52977eb8'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:36:37.162360 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-b38f6ff3-b24d-42d9-93dc-aaa92a461c54 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f43e14473d3848b58fcca06a52977eb8'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:36:37.174016 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-b38f6ff3-b24d-42d9-93dc-aaa92a461c54 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f43e14473d3848b58fcca06a52977eb8'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:36:37.179918 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 999/2014] 10.222.0.30 () {58 vars in 1224 bytes} [Tue Sep 6 04:36:36 2022] DELETE /identity/v3/users/f43e14473d3848b58fcca06a52977eb8 => generated 0 bytes in 410 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 04:36:37.184569 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc78146d-bc30-4762-a640-2c7d0c537eb5 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:36:37.185070 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc78146d-bc30-4762-a640-2c7d0c537eb5 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:36:37.185170 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc78146d-bc30-4762-a640-2c7d0c537eb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:36:37.205641 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-fc78146d-bc30-4762-a640-2c7d0c537eb5 None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:36:37.206861 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-fc78146d-bc30-4762-a640-2c7d0c537eb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:36:37.234753 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38f6ff3-b24d-42d9-93dc-aaa92a461c54 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:36:37.236314 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-b38f6ff3-b24d-42d9-93dc-aaa92a461c54 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:36:37.279700 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1016/2015] 10.222.0.30 () {58 vars in 958 bytes} [Tue Sep 6 04:36:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 04:36:37.291628 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc78146d-bc30-4762-a640-2c7d0c537eb5 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:36:37.293122 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-fc78146d-bc30-4762-a640-2c7d0c537eb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:36:37.318586 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38f6ff3-b24d-42d9-93dc-aaa92a461c54 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:36:37.323315 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023e2711-8ec4-4184-9c6c-9fac71b31505 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:36:37.323969 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023e2711-8ec4-4184-9c6c-9fac71b31505 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:36:37.324251 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023e2711-8ec4-4184-9c6c-9fac71b31505 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:36:37.324526 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023e2711-8ec4-4184-9c6c-9fac71b31505 admin admin] PATH_INFO: `/v3/users/94407dcea4ea4222852b656fc1c7a20b` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:36:37.325805 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023e2711-8ec4-4184-9c6c-9fac71b31505 admin admin] RBAC: Authorizing `identity:delete_user(user_id=94407dcea4ea4222852b656fc1c7a20b)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:36:37.327730 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023e2711-8ec4-4184-9c6c-9fac71b31505 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:36:37.376630 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc78146d-bc30-4762-a640-2c7d0c537eb5 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:36:37.383094 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4efab9c4-af08-417e-b0c4-e8e6407a0786 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90492440f6a6471e8392361c098428aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b707733ce65c42edbc1fd95ad0a1c06c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:36:37.384066 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efab9c4-af08-417e-b0c4-e8e6407a0786 service neutron] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:36:37.384366 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efab9c4-af08-417e-b0c4-e8e6407a0786 service neutron] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:36:37.384611 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efab9c4-af08-417e-b0c4-e8e6407a0786 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:36:37.396847 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4efab9c4-af08-417e-b0c4-e8e6407a0786 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:36:37.398460 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4efab9c4-af08-417e-b0c4-e8e6407a0786 service neutron] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:36:37.405240 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:36:37.411483 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1017/2016] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Sep 6 04:36:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 04:36:37.564209 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-023e2711-8ec4-4184-9c6c-9fac71b31505 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '94407dcea4ea4222852b656fc1c7a20b'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:36:37.652304 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-023e2711-8ec4-4184-9c6c-9fac71b31505 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '94407dcea4ea4222852b656fc1c7a20b'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:36:37.653015 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-023e2711-8ec4-4184-9c6c-9fac71b31505 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '94407dcea4ea4222852b656fc1c7a20b'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:36:37.669476 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-023e2711-8ec4-4184-9c6c-9fac71b31505 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '94407dcea4ea4222852b656fc1c7a20b'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:36:37.675399 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:36:37.683117 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-023e2711-8ec4-4184-9c6c-9fac71b31505 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '94407dcea4ea4222852b656fc1c7a20b'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:36:37.692377 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1000/2017] 10.222.0.30 () {58 vars in 1224 bytes} [Tue Sep 6 04:36:37 2022] DELETE /identity/v3/users/94407dcea4ea4222852b656fc1c7a20b => generated 0 bytes in 458 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 04:36:37.697328 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02763ba1-f4fa-4fae-ad35-c4c1e8effa4e None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:36:37.697715 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02763ba1-f4fa-4fae-ad35-c4c1e8effa4e None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:36:37.697994 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02763ba1-f4fa-4fae-ad35-c4c1e8effa4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:36:37.720337 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-02763ba1-f4fa-4fae-ad35-c4c1e8effa4e None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:36:37.721592 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-02763ba1-f4fa-4fae-ad35-c4c1e8effa4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:36:37.792727 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1018/2018] 10.222.0.30 () {58 vars in 958 bytes} [Tue Sep 6 04:36:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 04:36:37.803196 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023e2711-8ec4-4184-9c6c-9fac71b31505 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:36:37.804716 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-023e2711-8ec4-4184-9c6c-9fac71b31505 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:36:37.865144 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023e2711-8ec4-4184-9c6c-9fac71b31505 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:36:37.868725 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2823564-a5eb-4251-a445-fe7c4702184b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90492440f6a6471e8392361c098428aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b707733ce65c42edbc1fd95ad0a1c06c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:36:37.869424 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2823564-a5eb-4251-a445-fe7c4702184b service neutron] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:36:37.869710 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2823564-a5eb-4251-a445-fe7c4702184b service neutron] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:36:37.869986 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2823564-a5eb-4251-a445-fe7c4702184b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:36:37.879511 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2823564-a5eb-4251-a445-fe7c4702184b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:36:37.880943 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2823564-a5eb-4251-a445-fe7c4702184b service neutron] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:36:37.890412 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1001/2019] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Sep 6 04:36:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 04:36:37.933237 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02763ba1-f4fa-4fae-ad35-c4c1e8effa4e None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:36:37.934826 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-02763ba1-f4fa-4fae-ad35-c4c1e8effa4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:36:37.958101 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02763ba1-f4fa-4fae-ad35-c4c1e8effa4e None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:36:37.966796 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab3c37c-849b-498f-b927-771393140887 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:36:37.966796 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab3c37c-849b-498f-b927-771393140887 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:36:37.966796 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab3c37c-849b-498f-b927-771393140887 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:36:37.966796 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab3c37c-849b-498f-b927-771393140887 admin admin] PATH_INFO: `/v3/projects/206a2b9767ec4997904d37fd41624f25` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:36:37.967136 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab3c37c-849b-498f-b927-771393140887 admin admin] RBAC: Authorizing `identity:delete_project(project_id=206a2b9767ec4997904d37fd41624f25)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:36:37.968280 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab3c37c-849b-498f-b927-771393140887 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:36:38.092556 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-eab3c37c-849b-498f-b927-771393140887 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '206a2b9767ec4997904d37fd41624f25', 'request_id': 'req-eab3c37c-849b-498f-b927-771393140887'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:36:38.098488 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-eab3c37c-849b-498f-b927-771393140887 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '206a2b9767ec4997904d37fd41624f25', 'request_id': 'req-eab3c37c-849b-498f-b927-771393140887'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:36:38.171671 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-eab3c37c-849b-498f-b927-771393140887 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '206a2b9767ec4997904d37fd41624f25', 'request_id': 'req-eab3c37c-849b-498f-b927-771393140887'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:36:38.185133 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-eab3c37c-849b-498f-b927-771393140887 admin admin] The token cache is being invalidate because project 206a2b9767ec4997904d37fd41624f25 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81698) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 04:36:38.185755 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-eab3c37c-849b-498f-b927-771393140887 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:36:38.186410 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-eab3c37c-849b-498f-b927-771393140887 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:36:38.192644 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1019/2020] 10.222.0.30 () {58 vars in 1233 bytes} [Tue Sep 6 04:36:37 2022] DELETE /identity/v3/projects/206a2b9767ec4997904d37fd41624f25 => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 04:36:38.195013 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b27c07-bbb2-4ab3-b2ef-5197ed357316 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:36:38.195458 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b27c07-bbb2-4ab3-b2ef-5197ed357316 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:36:38.195834 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b27c07-bbb2-4ab3-b2ef-5197ed357316 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:36:38.228479 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-d8b27c07-bbb2-4ab3-b2ef-5197ed357316 None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:36:38.230015 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-d8b27c07-bbb2-4ab3-b2ef-5197ed357316 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:36:38.319264 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1002/2021] 10.222.0.30 () {58 vars in 958 bytes} [Tue Sep 6 04:36:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 04:36:38.333065 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3178bc-65dc-4d11-bd99-96ac512f8233 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:36:38.333713 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3178bc-65dc-4d11-bd99-96ac512f8233 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:36:38.334074 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3178bc-65dc-4d11-bd99-96ac512f8233 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:36:38.357298 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-7c3178bc-65dc-4d11-bd99-96ac512f8233 None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:36:38.359099 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-7c3178bc-65dc-4d11-bd99-96ac512f8233 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:36:38.368324 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1020/2022] 10.222.0.30 () {58 vars in 958 bytes} [Tue Sep 6 04:36:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 04:36:38.372610 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b27c07-bbb2-4ab3-b2ef-5197ed357316 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:36:38.386236 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b27c07-bbb2-4ab3-b2ef-5197ed357316 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:36:38.392080 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392a832c-dbac-4bd9-83f2-445c42c684d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:36:38.392988 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392a832c-dbac-4bd9-83f2-445c42c684d5 admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:36:38.393375 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392a832c-dbac-4bd9-83f2-445c42c684d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:36:38.393721 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392a832c-dbac-4bd9-83f2-445c42c684d5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:36:38.394452 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392a832c-dbac-4bd9-83f2-445c42c684d5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:36:38.394825 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392a832c-dbac-4bd9-83f2-445c42c684d5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:36:38.396498 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392a832c-dbac-4bd9-83f2-445c42c684d5 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:36:38.410078 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1003/2023] 10.222.0.30 () {58 vars in 1153 bytes} [Tue Sep 6 04:36:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:36:38.414190 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3178bc-65dc-4d11-bd99-96ac512f8233 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:36:38.424841 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3178bc-65dc-4d11-bd99-96ac512f8233 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:36:38.429125 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392a832c-dbac-4bd9-83f2-445c42c684d5 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:36:38.430110 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cfd7e35-e937-4775-aa77-6df23d7dd51c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:36:38.430947 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfd7e35-e937-4775-aa77-6df23d7dd51c admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:36:38.431232 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfd7e35-e937-4775-aa77-6df23d7dd51c admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:36:38.431644 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfd7e35-e937-4775-aa77-6df23d7dd51c admin admin] PATH_INFO: `/v3/projects` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:36:38.432144 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-392a832c-dbac-4bd9-83f2-445c42c684d5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:36:38.432307 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cfd7e35-e937-4775-aa77-6df23d7dd51c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:36:38.434482 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cfd7e35-e937-4775-aa77-6df23d7dd51c admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:36:38.449224 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392a832c-dbac-4bd9-83f2-445c42c684d5 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:36:38.462739 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab63e63d-5a1d-4139-aee5-fc1c3d5006f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:36:38.464290 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab63e63d-5a1d-4139-aee5-fc1c3d5006f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:36:38.465137 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab63e63d-5a1d-4139-aee5-fc1c3d5006f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:36:38.465602 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab63e63d-5a1d-4139-aee5-fc1c3d5006f3 admin admin] PATH_INFO: `/v3/projects/e4ffc44366814a1eaaf99740e6e66808` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:36:38.471241 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab63e63d-5a1d-4139-aee5-fc1c3d5006f3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e4ffc44366814a1eaaf99740e6e66808)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:36:38.474126 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab63e63d-5a1d-4139-aee5-fc1c3d5006f3 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:36:38.568999 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1021/2024] 10.222.0.30 () {60 vars in 1153 bytes} [Tue Sep 6 04:36:38 2022] POST /identity/v3/projects => generated 391 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 04:36:38.573081 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cfd7e35-e937-4775-aa77-6df23d7dd51c admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:36:38.620693 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-ab63e63d-5a1d-4139-aee5-fc1c3d5006f3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e4ffc44366814a1eaaf99740e6e66808', 'request_id': 'req-ab63e63d-5a1d-4139-aee5-fc1c3d5006f3'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:36:38.671896 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:36:38.679533 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cfd7e35-e937-4775-aa77-6df23d7dd51c admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:36:38.719514 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-ab63e63d-5a1d-4139-aee5-fc1c3d5006f3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e4ffc44366814a1eaaf99740e6e66808', 'request_id': 'req-ab63e63d-5a1d-4139-aee5-fc1c3d5006f3'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:36:38.726490 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-ab63e63d-5a1d-4139-aee5-fc1c3d5006f3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e4ffc44366814a1eaaf99740e6e66808', 'request_id': 'req-ab63e63d-5a1d-4139-aee5-fc1c3d5006f3'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:36:38.744176 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:36:38.757123 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-ab63e63d-5a1d-4139-aee5-fc1c3d5006f3 admin admin] The token cache is being invalidate because project e4ffc44366814a1eaaf99740e6e66808 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81697) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 04:36:38.757594 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-ab63e63d-5a1d-4139-aee5-fc1c3d5006f3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:36:38.759510 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-ab63e63d-5a1d-4139-aee5-fc1c3d5006f3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:36:38.762432 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1004/2025] 10.222.0.30 () {58 vars in 1233 bytes} [Tue Sep 6 04:36:38 2022] DELETE /identity/v3/projects/e4ffc44366814a1eaaf99740e6e66808 => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 04:36:38.766613 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab63e63d-5a1d-4139-aee5-fc1c3d5006f3 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:36:38.774015 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-ab63e63d-5a1d-4139-aee5-fc1c3d5006f3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:36:38.814685 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ec3e33-b16c-4362-9890-0eb311cf0d00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:36:38.815322 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ec3e33-b16c-4362-9890-0eb311cf0d00 admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:36:38.815606 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ec3e33-b16c-4362-9890-0eb311cf0d00 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:36:38.815944 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ec3e33-b16c-4362-9890-0eb311cf0d00 admin admin] PATH_INFO: `/v3/users` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:36:38.816639 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ec3e33-b16c-4362-9890-0eb311cf0d00 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:36:38.818819 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ec3e33-b16c-4362-9890-0eb311cf0d00 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:36:38.874414 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab63e63d-5a1d-4139-aee5-fc1c3d5006f3 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:36:38.894492 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970e9ea8-a0c5-447d-8ba0-373ac5e65359 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:36:38.895886 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970e9ea8-a0c5-447d-8ba0-373ac5e65359 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:36:38.896174 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970e9ea8-a0c5-447d-8ba0-373ac5e65359 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:36:38.896455 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970e9ea8-a0c5-447d-8ba0-373ac5e65359 admin admin] PATH_INFO: `/v3/users/b9e73262a85e4956b25be88a9ae2a976` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:36:38.898653 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-970e9ea8-a0c5-447d-8ba0-373ac5e65359 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b9e73262a85e4956b25be88a9ae2a976)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:36:38.900892 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-970e9ea8-a0c5-447d-8ba0-373ac5e65359 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:36:38.912934 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1022/2026] 10.222.0.30 () {60 vars in 1144 bytes} [Tue Sep 6 04:36:38 2022] POST /identity/v3/users => generated 296 bytes in 340 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 04:36:38.917292 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ec3e33-b16c-4362-9890-0eb311cf0d00 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:36:38.919738 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-34ec3e33-b16c-4362-9890-0eb311cf0d00 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:36:38.933534 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ec3e33-b16c-4362-9890-0eb311cf0d00 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:36:38.939010 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f6b3e9-2baa-4e32-b3c7-561ef23aa2f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:36:38.939868 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f6b3e9-2baa-4e32-b3c7-561ef23aa2f1 admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:36:38.940266 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f6b3e9-2baa-4e32-b3c7-561ef23aa2f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:36:38.940604 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f6b3e9-2baa-4e32-b3c7-561ef23aa2f1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:36:38.941347 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f6b3e9-2baa-4e32-b3c7-561ef23aa2f1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:36:38.943034 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f6b3e9-2baa-4e32-b3c7-561ef23aa2f1 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:36:38.954326 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1023/2027] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Sep 6 04:36:38 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 04:36:38.958414 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f6b3e9-2baa-4e32-b3c7-561ef23aa2f1 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:36:38.980151 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f6b3e9-2baa-4e32-b3c7-561ef23aa2f1 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:36:38.986671 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4d54f8-e52a-42c6-a3b9-fcb0a560af47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:36:38.987560 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4d54f8-e52a-42c6-a3b9-fcb0a560af47 admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:36:38.987843 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4d54f8-e52a-42c6-a3b9-fcb0a560af47 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:36:38.988119 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4d54f8-e52a-42c6-a3b9-fcb0a560af47 admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:36:38.988768 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4d54f8-e52a-42c6-a3b9-fcb0a560af47 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:36:38.990419 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4d54f8-e52a-42c6-a3b9-fcb0a560af47 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:36:39.002187 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1024/2028] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Sep 6 04:36:38 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 06 04:36:39.006601 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4d54f8-e52a-42c6-a3b9-fcb0a560af47 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:36:39.016738 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4d54f8-e52a-42c6-a3b9-fcb0a560af47 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:36:39.062369 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe972353-7d14-4a1c-86f7-fcce8a90bbd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:36:39.062369 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe972353-7d14-4a1c-86f7-fcce8a90bbd6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:36:39.062369 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe972353-7d14-4a1c-86f7-fcce8a90bbd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:36:39.062369 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe972353-7d14-4a1c-86f7-fcce8a90bbd6 admin admin] PATH_INFO: `/v3/projects/a43bf76a0ad84d77b41d0115f0729fc1/users/d14b04d1ae9748c89a46b2994625bcad/roles/29607b2de39f40d68bb6ceef64b8a732` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:36:39.063310 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe972353-7d14-4a1c-86f7-fcce8a90bbd6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a43bf76a0ad84d77b41d0115f0729fc1, user_id=d14b04d1ae9748c89a46b2994625bcad, role_id=29607b2de39f40d68bb6ceef64b8a732)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:36:39.063310 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe972353-7d14-4a1c-86f7-fcce8a90bbd6 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:36:39.212937 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1025/2029] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Sep 6 04:36:39 2022] PUT /identity/v3/projects/a43bf76a0ad84d77b41d0115f0729fc1/users/d14b04d1ae9748c89a46b2994625bcad/roles/29607b2de39f40d68bb6ceef64b8a732 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 04:36:39.223843 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d48060-8715-46cd-8216-eb161ebea1ee None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:36:39.224383 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d48060-8715-46cd-8216-eb161ebea1ee None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:36:39.225265 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d48060-8715-46cd-8216-eb161ebea1ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:36:39.256036 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-970e9ea8-a0c5-447d-8ba0-373ac5e65359 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b9e73262a85e4956b25be88a9ae2a976'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:36:39.264174 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-83d48060-8715-46cd-8216-eb161ebea1ee None None] MFA Rules not processed for user `d14b04d1ae9748c89a46b2994625bcad`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:36:39.268789 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-83d48060-8715-46cd-8216-eb161ebea1ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:36:39.315193 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-970e9ea8-a0c5-447d-8ba0-373ac5e65359 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b9e73262a85e4956b25be88a9ae2a976'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:36:39.315868 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-970e9ea8-a0c5-447d-8ba0-373ac5e65359 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b9e73262a85e4956b25be88a9ae2a976'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:36:39.325642 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-970e9ea8-a0c5-447d-8ba0-373ac5e65359 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b9e73262a85e4956b25be88a9ae2a976'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:36:39.346438 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1026/2030] 10.222.0.30 () {58 vars in 958 bytes} [Tue Sep 6 04:36:39 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 04:36:39.352124 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-970e9ea8-a0c5-447d-8ba0-373ac5e65359 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b9e73262a85e4956b25be88a9ae2a976'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:36:39.358523 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1005/2031] 10.222.0.30 () {58 vars in 1224 bytes} [Tue Sep 6 04:36:38 2022] DELETE /identity/v3/users/b9e73262a85e4956b25be88a9ae2a976 => generated 0 bytes in 593 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 04:36:39.433148 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d48060-8715-46cd-8216-eb161ebea1ee None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:36:39.434911 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-83d48060-8715-46cd-8216-eb161ebea1ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:36:39.490693 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:36:39.511312 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d48060-8715-46cd-8216-eb161ebea1ee None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:36:39.559775 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d59052d-c71d-4484-98b6-31ccf6f14832 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd09b5ab675634a2eb4b03755743f7d0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b707733ce65c42edbc1fd95ad0a1c06c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:36:39.560394 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d59052d-c71d-4484-98b6-31ccf6f14832 service cinder] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:36:39.560676 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d59052d-c71d-4484-98b6-31ccf6f14832 service cinder] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:36:39.561004 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d59052d-c71d-4484-98b6-31ccf6f14832 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:36:39.567045 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-3d59052d-c71d-4484-98b6-31ccf6f14832 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:36:39.665510 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d59052d-c71d-4484-98b6-31ccf6f14832 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:36:39.667258 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d59052d-c71d-4484-98b6-31ccf6f14832 service cinder] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:36:39.681324 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1027/2032] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Sep 6 04:36:39 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 249 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 04:36:39.860440 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970e9ea8-a0c5-447d-8ba0-373ac5e65359 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:36:39.862090 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-970e9ea8-a0c5-447d-8ba0-373ac5e65359 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:36:39.959061 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970e9ea8-a0c5-447d-8ba0-373ac5e65359 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:36:39.964261 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790d08ac-6880-4fe2-aa5c-605a25ef8d76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:36:39.965214 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790d08ac-6880-4fe2-aa5c-605a25ef8d76 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:36:39.965649 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790d08ac-6880-4fe2-aa5c-605a25ef8d76 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:36:39.966030 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790d08ac-6880-4fe2-aa5c-605a25ef8d76 admin admin] PATH_INFO: `/v3/projects/77fde6d9ba394d79adba96ed3d3b7aed` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:36:39.967305 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790d08ac-6880-4fe2-aa5c-605a25ef8d76 admin admin] RBAC: Authorizing `identity:delete_project(project_id=77fde6d9ba394d79adba96ed3d3b7aed)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:36:39.969734 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790d08ac-6880-4fe2-aa5c-605a25ef8d76 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:36:40.001090 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:36:40.095009 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-790d08ac-6880-4fe2-aa5c-605a25ef8d76 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '77fde6d9ba394d79adba96ed3d3b7aed', 'request_id': 'req-790d08ac-6880-4fe2-aa5c-605a25ef8d76'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:36:40.161409 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-790d08ac-6880-4fe2-aa5c-605a25ef8d76 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '77fde6d9ba394d79adba96ed3d3b7aed', 'request_id': 'req-790d08ac-6880-4fe2-aa5c-605a25ef8d76'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:36:40.168161 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-790d08ac-6880-4fe2-aa5c-605a25ef8d76 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '77fde6d9ba394d79adba96ed3d3b7aed', 'request_id': 'req-790d08ac-6880-4fe2-aa5c-605a25ef8d76'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:36:40.178721 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-790d08ac-6880-4fe2-aa5c-605a25ef8d76 admin admin] The token cache is being invalidate because project 77fde6d9ba394d79adba96ed3d3b7aed was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81697) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 04:36:40.179008 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-790d08ac-6880-4fe2-aa5c-605a25ef8d76 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:36:40.179644 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-790d08ac-6880-4fe2-aa5c-605a25ef8d76 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:36:40.181113 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1006/2033] 10.222.0.30 () {58 vars in 1233 bytes} [Tue Sep 6 04:36:39 2022] DELETE /identity/v3/projects/77fde6d9ba394d79adba96ed3d3b7aed => generated 0 bytes in 321 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 04:36:40.187141 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e036ee1a-8ea3-4d6c-a540-071684f41621 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:36:40.189429 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e036ee1a-8ea3-4d6c-a540-071684f41621 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:36:40.189429 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e036ee1a-8ea3-4d6c-a540-071684f41621 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:36:40.208966 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-e036ee1a-8ea3-4d6c-a540-071684f41621 None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:36:40.210462 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-e036ee1a-8ea3-4d6c-a540-071684f41621 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:36:40.275927 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1028/2034] 10.222.0.30 () {58 vars in 958 bytes} [Tue Sep 6 04:36:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 04:36:40.285205 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02667fe7-4080-47d7-8bda-705cfff0b151 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:36:40.285453 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02667fe7-4080-47d7-8bda-705cfff0b151 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:36:40.285664 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02667fe7-4080-47d7-8bda-705cfff0b151 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:36:40.313449 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-02667fe7-4080-47d7-8bda-705cfff0b151 None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:36:40.314740 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-02667fe7-4080-47d7-8bda-705cfff0b151 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:36:40.322953 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1007/2035] 10.222.0.30 () {58 vars in 958 bytes} [Tue Sep 6 04:36:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 04:36:40.327244 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e036ee1a-8ea3-4d6c-a540-071684f41621 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:36:40.337511 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e036ee1a-8ea3-4d6c-a540-071684f41621 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:36:40.342366 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee869e2-db06-4ac9-8428-7efa8654bc42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:36:40.342994 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee869e2-db06-4ac9-8428-7efa8654bc42 admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:36:40.343404 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee869e2-db06-4ac9-8428-7efa8654bc42 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:36:40.343744 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee869e2-db06-4ac9-8428-7efa8654bc42 admin admin] PATH_INFO: `/v3/domains` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:36:40.344485 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee869e2-db06-4ac9-8428-7efa8654bc42 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:36:40.344783 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee869e2-db06-4ac9-8428-7efa8654bc42 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:36:40.346426 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee869e2-db06-4ac9-8428-7efa8654bc42 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:36:40.362656 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1029/2036] 10.222.0.30 () {58 vars in 1153 bytes} [Tue Sep 6 04:36:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:36:40.366351 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02667fe7-4080-47d7-8bda-705cfff0b151 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:36:40.389978 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02667fe7-4080-47d7-8bda-705cfff0b151 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:36:40.395134 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb545ef-afd0-458a-9fba-254ea4d96dde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:36:40.396896 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb545ef-afd0-458a-9fba-254ea4d96dde admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:36:40.399840 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb545ef-afd0-458a-9fba-254ea4d96dde admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:36:40.399840 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb545ef-afd0-458a-9fba-254ea4d96dde admin admin] PATH_INFO: `/v3/projects` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:36:40.399840 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb545ef-afd0-458a-9fba-254ea4d96dde admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:36:40.400518 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb545ef-afd0-458a-9fba-254ea4d96dde admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:36:40.516660 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1008/2037] 10.222.0.30 () {60 vars in 1153 bytes} [Tue Sep 6 04:36:40 2022] POST /identity/v3/projects => generated 377 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 04:36:40.520896 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee869e2-db06-4ac9-8428-7efa8654bc42 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:36:40.531681 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee869e2-db06-4ac9-8428-7efa8654bc42 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:36:40.561172 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:36:40.587119 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e9ca99-8d57-4e16-a4ee-82af3300f5fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:36:40.587745 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e9ca99-8d57-4e16-a4ee-82af3300f5fb admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:36:40.588002 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e9ca99-8d57-4e16-a4ee-82af3300f5fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:36:40.588268 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e9ca99-8d57-4e16-a4ee-82af3300f5fb admin admin] PATH_INFO: `/v3/users` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:36:40.588884 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e9ca99-8d57-4e16-a4ee-82af3300f5fb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:36:40.590890 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e9ca99-8d57-4e16-a4ee-82af3300f5fb admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:36:40.682114 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1030/2038] 10.222.0.30 () {60 vars in 1144 bytes} [Tue Sep 6 04:36:40 2022] POST /identity/v3/users => generated 289 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 04:36:40.685411 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb545ef-afd0-458a-9fba-254ea4d96dde admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:36:40.696258 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb545ef-afd0-458a-9fba-254ea4d96dde admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:36:40.700948 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6e6947-4061-4802-b668-c5c11218f55b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:36:40.701584 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6e6947-4061-4802-b668-c5c11218f55b admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:36:40.701862 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6e6947-4061-4802-b668-c5c11218f55b admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:36:40.702129 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6e6947-4061-4802-b668-c5c11218f55b admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:36:40.702841 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b6e6947-4061-4802-b668-c5c11218f55b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:36:40.704579 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b6e6947-4061-4802-b668-c5c11218f55b admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:36:40.714775 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1009/2039] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Sep 6 04:36:40 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 04:36:40.718553 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e9ca99-8d57-4e16-a4ee-82af3300f5fb admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:36:40.729353 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e9ca99-8d57-4e16-a4ee-82af3300f5fb admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:36:40.734114 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586713ab-4c28-4b70-96bf-cfaf37e4b9eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:36:40.735632 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586713ab-4c28-4b70-96bf-cfaf37e4b9eb admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:36:40.735940 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586713ab-4c28-4b70-96bf-cfaf37e4b9eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:36:40.736215 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586713ab-4c28-4b70-96bf-cfaf37e4b9eb admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:36:40.736865 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586713ab-4c28-4b70-96bf-cfaf37e4b9eb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:36:40.738495 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586713ab-4c28-4b70-96bf-cfaf37e4b9eb admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:36:40.749480 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1031/2040] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Sep 6 04:36:40 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 04:36:40.754112 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6e6947-4061-4802-b668-c5c11218f55b admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:36:40.764335 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6e6947-4061-4802-b668-c5c11218f55b admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:36:40.769449 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138c86c6-a985-4bca-a09d-4fe55139a5cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:36:40.770099 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138c86c6-a985-4bca-a09d-4fe55139a5cc admin admin] REQUEST_METHOD: `PUT` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:36:40.770406 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138c86c6-a985-4bca-a09d-4fe55139a5cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:36:40.770680 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138c86c6-a985-4bca-a09d-4fe55139a5cc admin admin] PATH_INFO: `/v3/projects/5eded26134634335ab9ed9afc6067854/users/b5ff6cccb0fd4573b5cfa22d808e3392/roles/29607b2de39f40d68bb6ceef64b8a732` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:36:40.785954 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138c86c6-a985-4bca-a09d-4fe55139a5cc admin admin] RBAC: Authorizing `identity:create_grant(project_id=5eded26134634335ab9ed9afc6067854, user_id=b5ff6cccb0fd4573b5cfa22d808e3392, role_id=29607b2de39f40d68bb6ceef64b8a732)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:36:40.788560 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138c86c6-a985-4bca-a09d-4fe55139a5cc admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:36:40.885226 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1010/2041] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Sep 6 04:36:40 2022] PUT /identity/v3/projects/5eded26134634335ab9ed9afc6067854/users/b5ff6cccb0fd4573b5cfa22d808e3392/roles/29607b2de39f40d68bb6ceef64b8a732 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 04:36:40.898145 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5dc4efc-2520-4653-912f-a0bbcf18cc95 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:36:40.898462 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5dc4efc-2520-4653-912f-a0bbcf18cc95 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:36:40.898742 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5dc4efc-2520-4653-912f-a0bbcf18cc95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:36:40.927496 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-c5dc4efc-2520-4653-912f-a0bbcf18cc95 None None] MFA Rules not processed for user `b5ff6cccb0fd4573b5cfa22d808e3392`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:36:40.928709 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-c5dc4efc-2520-4653-912f-a0bbcf18cc95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:36:40.992490 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1032/2042] 10.222.0.30 () {58 vars in 958 bytes} [Tue Sep 6 04:36:40 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 04:36:41.003459 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138c86c6-a985-4bca-a09d-4fe55139a5cc admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:36:41.005115 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-138c86c6-a985-4bca-a09d-4fe55139a5cc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:36:41.064583 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138c86c6-a985-4bca-a09d-4fe55139a5cc admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:36:41.068775 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90a4982-b01e-4ead-b764-9b439c4a92bc service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae5c31fb1d0342c0937bbc83a415113d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b707733ce65c42edbc1fd95ad0a1c06c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:36:41.069499 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90a4982-b01e-4ead-b764-9b439c4a92bc service glance] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:36:41.069786 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90a4982-b01e-4ead-b764-9b439c4a92bc service glance] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:36:41.070065 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90a4982-b01e-4ead-b764-9b439c4a92bc service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:36:41.079842 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90a4982-b01e-4ead-b764-9b439c4a92bc service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:36:41.081445 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90a4982-b01e-4ead-b764-9b439c4a92bc service glance] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:36:41.098205 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1011/2043] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Sep 6 04:36:41 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 04:36:50.989579 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=51,keystone:INSERT=3 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:36:51.086661 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:DELETE=7,keystone:SELECT=68,keystone:INSERT=3 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:36:51.492987 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393929cf-5c8d-4cba-9a20-6bc71a1a1046 None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:36:51.493694 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393929cf-5c8d-4cba-9a20-6bc71a1a1046 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:36:51.493986 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393929cf-5c8d-4cba-9a20-6bc71a1a1046 None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:36:51.495634 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1033/2044] 10.222.0.30 () {54 vars in 921 bytes} [Tue Sep 6 04:36:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 04:36:51.534171 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cfd0d9-0a0f-44f3-ac24-62169c19707c None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:36:51.535391 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cfd0d9-0a0f-44f3-ac24-62169c19707c None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:36:51.535950 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cfd0d9-0a0f-44f3-ac24-62169c19707c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:36:51.562362 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-e4cfd0d9-0a0f-44f3-ac24-62169c19707c None None] MFA Rules not processed for user `1d4f2e4dbd41441d84f03ddb30191e7a`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:36:51.564078 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-e4cfd0d9-0a0f-44f3-ac24-62169c19707c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:36:51.646545 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1012/2045] 10.222.0.30 () {58 vars in 1018 bytes} [Tue Sep 6 04:36:51 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 04:36:51.660067 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393929cf-5c8d-4cba-9a20-6bc71a1a1046 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:36:51.665593 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-393929cf-5c8d-4cba-9a20-6bc71a1a1046 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:36:51.672098 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4cfd0d9-0a0f-44f3-ac24-62169c19707c None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:36:51.673724 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-e4cfd0d9-0a0f-44f3-ac24-62169c19707c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:36:51.697067 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393929cf-5c8d-4cba-9a20-6bc71a1a1046 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:36:51.702934 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cdf5f9e-3488-4e1f-9bff-4d5b6ee8fcca service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d4f2e4dbd41441d84f03ddb30191e7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b707733ce65c42edbc1fd95ad0a1c06c', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:36:51.703834 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdf5f9e-3488-4e1f-9bff-4d5b6ee8fcca service placement] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:36:51.704372 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdf5f9e-3488-4e1f-9bff-4d5b6ee8fcca service placement] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:36:51.704881 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdf5f9e-3488-4e1f-9bff-4d5b6ee8fcca service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:36:51.716338 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cdf5f9e-3488-4e1f-9bff-4d5b6ee8fcca service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:36:51.718206 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cdf5f9e-3488-4e1f-9bff-4d5b6ee8fcca service placement] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:36:51.730629 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1034/2046] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Sep 6 04:36:51 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 71 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 04:36:51.764267 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4cfd0d9-0a0f-44f3-ac24-62169c19707c None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:36:51.770709 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5758fc-a673-4b9a-a6b2-3f19244ae66d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:36:51.771623 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5758fc-a673-4b9a-a6b2-3f19244ae66d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:36:51.771912 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5758fc-a673-4b9a-a6b2-3f19244ae66d admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:36:51.772186 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5758fc-a673-4b9a-a6b2-3f19244ae66d admin admin] PATH_INFO: `/v3/users/15eebbd8b1d44f4897acb266633998db` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:36:51.774119 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5758fc-a673-4b9a-a6b2-3f19244ae66d admin admin] RBAC: Authorizing `identity:delete_user(user_id=15eebbd8b1d44f4897acb266633998db)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:36:51.777694 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5758fc-a673-4b9a-a6b2-3f19244ae66d admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:36:52.002855 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-6b5758fc-a673-4b9a-a6b2-3f19244ae66d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '15eebbd8b1d44f4897acb266633998db'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:36:52.074856 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-6b5758fc-a673-4b9a-a6b2-3f19244ae66d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '15eebbd8b1d44f4897acb266633998db'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:36:52.075498 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-6b5758fc-a673-4b9a-a6b2-3f19244ae66d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '15eebbd8b1d44f4897acb266633998db'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:36:52.083138 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-6b5758fc-a673-4b9a-a6b2-3f19244ae66d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '15eebbd8b1d44f4897acb266633998db'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:36:52.100040 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-6b5758fc-a673-4b9a-a6b2-3f19244ae66d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '15eebbd8b1d44f4897acb266633998db'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:36:52.106113 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1013/2047] 10.222.0.30 () {58 vars in 1224 bytes} [Tue Sep 6 04:36:51 2022] DELETE /identity/v3/users/15eebbd8b1d44f4897acb266633998db => generated 0 bytes in 435 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 04:36:52.358217 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cdf5f9e-3488-4e1f-9bff-4d5b6ee8fcca service placement] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:36:52.369938 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cdf5f9e-3488-4e1f-9bff-4d5b6ee8fcca service placement] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:36:52.434954 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af9041e-8726-4d1b-b286-f506993ecf7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:36:52.435596 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af9041e-8726-4d1b-b286-f506993ecf7d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:36:52.435880 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af9041e-8726-4d1b-b286-f506993ecf7d admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:36:52.436154 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af9041e-8726-4d1b-b286-f506993ecf7d admin admin] PATH_INFO: `/v3/projects/96ba8ad37f17481a9691031fefbc3c80` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:36:52.437329 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af9041e-8726-4d1b-b286-f506993ecf7d admin admin] RBAC: Authorizing `identity:delete_project(project_id=96ba8ad37f17481a9691031fefbc3c80)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:36:52.439279 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af9041e-8726-4d1b-b286-f506993ecf7d admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:36:52.549592 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-1af9041e-8726-4d1b-b286-f506993ecf7d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '96ba8ad37f17481a9691031fefbc3c80', 'request_id': 'req-1af9041e-8726-4d1b-b286-f506993ecf7d'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:36:52.556469 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-1af9041e-8726-4d1b-b286-f506993ecf7d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '96ba8ad37f17481a9691031fefbc3c80', 'request_id': 'req-1af9041e-8726-4d1b-b286-f506993ecf7d'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:36:52.663628 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-1af9041e-8726-4d1b-b286-f506993ecf7d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '96ba8ad37f17481a9691031fefbc3c80', 'request_id': 'req-1af9041e-8726-4d1b-b286-f506993ecf7d'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:36:52.676191 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-1af9041e-8726-4d1b-b286-f506993ecf7d admin admin] The token cache is being invalidate because project 96ba8ad37f17481a9691031fefbc3c80 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81698) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 04:36:52.677257 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-1af9041e-8726-4d1b-b286-f506993ecf7d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:36:52.678727 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-1af9041e-8726-4d1b-b286-f506993ecf7d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:36:52.680998 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1035/2048] 10.222.0.30 () {58 vars in 1233 bytes} [Tue Sep 6 04:36:52 2022] DELETE /identity/v3/projects/96ba8ad37f17481a9691031fefbc3c80 => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 04:36:52.686053 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121b52f4-4134-4672-8bc9-4e5524664063 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:36:52.690061 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121b52f4-4134-4672-8bc9-4e5524664063 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:36:52.690061 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121b52f4-4134-4672-8bc9-4e5524664063 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:36:52.710019 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-121b52f4-4134-4672-8bc9-4e5524664063 None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:36:52.713023 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-121b52f4-4134-4672-8bc9-4e5524664063 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:36:52.721038 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:36:52.774274 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1014/2049] 10.222.0.30 () {58 vars in 958 bytes} [Tue Sep 6 04:36:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 04:36:52.782315 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce779af4-94a1-4985-9f19-c665aa587fe1 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:36:52.782617 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce779af4-94a1-4985-9f19-c665aa587fe1 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:36:52.782889 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce779af4-94a1-4985-9f19-c665aa587fe1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:36:52.803410 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-ce779af4-94a1-4985-9f19-c665aa587fe1 None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:36:52.804699 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-ce779af4-94a1-4985-9f19-c665aa587fe1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:36:52.813013 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1036/2050] 10.222.0.30 () {58 vars in 958 bytes} [Tue Sep 6 04:36:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 04:36:52.817973 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121b52f4-4134-4672-8bc9-4e5524664063 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:36:52.827535 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121b52f4-4134-4672-8bc9-4e5524664063 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:36:52.832218 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1efbe37-1a8b-4940-b1b9-9fe70180cd17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:36:52.832822 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1efbe37-1a8b-4940-b1b9-9fe70180cd17 admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:36:52.833105 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1efbe37-1a8b-4940-b1b9-9fe70180cd17 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:36:52.835019 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1efbe37-1a8b-4940-b1b9-9fe70180cd17 admin admin] PATH_INFO: `/v3/domains` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:36:52.835019 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1efbe37-1a8b-4940-b1b9-9fe70180cd17 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:36:52.835019 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1efbe37-1a8b-4940-b1b9-9fe70180cd17 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:36:52.835926 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1efbe37-1a8b-4940-b1b9-9fe70180cd17 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:36:52.847447 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1015/2051] 10.222.0.30 () {58 vars in 1153 bytes} [Tue Sep 6 04:36:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:36:52.851775 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce779af4-94a1-4985-9f19-c665aa587fe1 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:36:52.861261 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce779af4-94a1-4985-9f19-c665aa587fe1 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:36:52.866839 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7825ac-1242-4504-b9f0-e982cf804982 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:36:52.867437 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7825ac-1242-4504-b9f0-e982cf804982 admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:36:52.868105 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7825ac-1242-4504-b9f0-e982cf804982 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:36:52.868105 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7825ac-1242-4504-b9f0-e982cf804982 admin admin] PATH_INFO: `/v3/projects` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:36:52.868650 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7825ac-1242-4504-b9f0-e982cf804982 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:36:52.870557 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7825ac-1242-4504-b9f0-e982cf804982 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:36:52.997254 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1037/2052] 10.222.0.30 () {60 vars in 1153 bytes} [Tue Sep 6 04:36:52 2022] POST /identity/v3/projects => generated 385 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 04:36:53.001438 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1efbe37-1a8b-4940-b1b9-9fe70180cd17 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:36:53.012158 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1efbe37-1a8b-4940-b1b9-9fe70180cd17 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:36:53.080247 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d18a528-2cba-4641-9c91-07461972d7a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:36:53.080247 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d18a528-2cba-4641-9c91-07461972d7a3 admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:36:53.080247 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d18a528-2cba-4641-9c91-07461972d7a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:36:53.080247 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d18a528-2cba-4641-9c91-07461972d7a3 admin admin] PATH_INFO: `/v3/users` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:36:53.080647 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d18a528-2cba-4641-9c91-07461972d7a3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:36:53.082367 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d18a528-2cba-4641-9c91-07461972d7a3 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:36:53.218426 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1016/2053] 10.222.0.30 () {60 vars in 1144 bytes} [Tue Sep 6 04:36:53 2022] POST /identity/v3/users => generated 293 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 04:36:53.222351 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7825ac-1242-4504-b9f0-e982cf804982 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:36:53.233918 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7825ac-1242-4504-b9f0-e982cf804982 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:36:53.239064 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33decba-7f99-46fe-91fe-c1874e525537 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:36:53.239725 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33decba-7f99-46fe-91fe-c1874e525537 admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:36:53.240011 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33decba-7f99-46fe-91fe-c1874e525537 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:36:53.240287 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33decba-7f99-46fe-91fe-c1874e525537 admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:36:53.240972 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33decba-7f99-46fe-91fe-c1874e525537 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:36:53.242653 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33decba-7f99-46fe-91fe-c1874e525537 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:36:53.255323 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1038/2054] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Sep 6 04:36:53 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 04:36:53.258873 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d18a528-2cba-4641-9c91-07461972d7a3 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:36:53.272893 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d18a528-2cba-4641-9c91-07461972d7a3 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:36:53.279108 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1f82c8-d284-4e11-8e73-b117600fdfd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:36:53.279762 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1f82c8-d284-4e11-8e73-b117600fdfd3 admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:36:53.280046 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1f82c8-d284-4e11-8e73-b117600fdfd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:36:53.280322 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1f82c8-d284-4e11-8e73-b117600fdfd3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:36:53.280970 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1f82c8-d284-4e11-8e73-b117600fdfd3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:36:53.283277 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1f82c8-d284-4e11-8e73-b117600fdfd3 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:36:53.299319 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1017/2055] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Sep 6 04:36:53 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 04:36:53.303846 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33decba-7f99-46fe-91fe-c1874e525537 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:36:53.313999 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33decba-7f99-46fe-91fe-c1874e525537 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:36:53.319211 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0608ae25-ae6e-49ad-a908-84ad727f4f26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:36:53.319864 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0608ae25-ae6e-49ad-a908-84ad727f4f26 admin admin] REQUEST_METHOD: `PUT` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:36:53.320145 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0608ae25-ae6e-49ad-a908-84ad727f4f26 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:36:53.320420 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0608ae25-ae6e-49ad-a908-84ad727f4f26 admin admin] PATH_INFO: `/v3/projects/0f5ff30b7d70423a809d6c3701149255/users/88fd2421b7cf4bf78cd6fbc209660e30/roles/29607b2de39f40d68bb6ceef64b8a732` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:36:53.335452 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0608ae25-ae6e-49ad-a908-84ad727f4f26 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0f5ff30b7d70423a809d6c3701149255, user_id=88fd2421b7cf4bf78cd6fbc209660e30, role_id=29607b2de39f40d68bb6ceef64b8a732)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:36:53.338141 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0608ae25-ae6e-49ad-a908-84ad727f4f26 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:36:53.418678 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1039/2056] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Sep 6 04:36:53 2022] PUT /identity/v3/projects/0f5ff30b7d70423a809d6c3701149255/users/88fd2421b7cf4bf78cd6fbc209660e30/roles/29607b2de39f40d68bb6ceef64b8a732 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 04:36:53.429164 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04787e4-f55c-40e5-a637-cd19f3a504c5 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:36:53.429489 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04787e4-f55c-40e5-a637-cd19f3a504c5 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:36:53.429763 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04787e4-f55c-40e5-a637-cd19f3a504c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:36:53.460576 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-f04787e4-f55c-40e5-a637-cd19f3a504c5 None None] MFA Rules not processed for user `88fd2421b7cf4bf78cd6fbc209660e30`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:36:53.461856 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-f04787e4-f55c-40e5-a637-cd19f3a504c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:36:53.534618 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1018/2057] 10.222.0.30 () {58 vars in 958 bytes} [Tue Sep 6 04:36:53 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 04:36:53.545687 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0608ae25-ae6e-49ad-a908-84ad727f4f26 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:36:53.547520 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-0608ae25-ae6e-49ad-a908-84ad727f4f26 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:36:53.563023 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:36:53.626343 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04787e4-f55c-40e5-a637-cd19f3a504c5 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:36:53.627766 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-f04787e4-f55c-40e5-a637-cd19f3a504c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:36:53.636729 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0608ae25-ae6e-49ad-a908-84ad727f4f26 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:36:53.641426 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:36:53.642591 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2eecf1-95ea-45d7-8504-e7d1bf2640d2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9db138b77afa4b068244b8ef715404fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b707733ce65c42edbc1fd95ad0a1c06c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:36:53.643205 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2eecf1-95ea-45d7-8504-e7d1bf2640d2 service nova] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:36:53.643515 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2eecf1-95ea-45d7-8504-e7d1bf2640d2 service nova] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:36:53.643802 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2eecf1-95ea-45d7-8504-e7d1bf2640d2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:36:53.652358 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d2eecf1-95ea-45d7-8504-e7d1bf2640d2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:36:53.653987 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d2eecf1-95ea-45d7-8504-e7d1bf2640d2 service nova] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:36:53.665418 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1040/2058] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Sep 6 04:36:53 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 04:36:53.709726 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04787e4-f55c-40e5-a637-cd19f3a504c5 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:36:53.717740 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6344f49-6843-40bd-ad8d-edd2f82becbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:36:53.717740 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6344f49-6843-40bd-ad8d-edd2f82becbe admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:36:53.717740 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6344f49-6843-40bd-ad8d-edd2f82becbe admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:36:53.717740 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6344f49-6843-40bd-ad8d-edd2f82becbe admin admin] PATH_INFO: `/v3/users/ed694a74b8e540768ee5458fcb5ce649` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:36:53.718061 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6344f49-6843-40bd-ad8d-edd2f82becbe admin admin] RBAC: Authorizing `identity:delete_user(user_id=ed694a74b8e540768ee5458fcb5ce649)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:36:53.718878 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6344f49-6843-40bd-ad8d-edd2f82becbe admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:36:53.950202 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-c6344f49-6843-40bd-ad8d-edd2f82becbe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ed694a74b8e540768ee5458fcb5ce649'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:36:54.056843 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-c6344f49-6843-40bd-ad8d-edd2f82becbe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ed694a74b8e540768ee5458fcb5ce649'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:36:54.057319 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-c6344f49-6843-40bd-ad8d-edd2f82becbe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ed694a74b8e540768ee5458fcb5ce649'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:36:54.065229 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-c6344f49-6843-40bd-ad8d-edd2f82becbe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ed694a74b8e540768ee5458fcb5ce649'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:36:54.078870 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-c6344f49-6843-40bd-ad8d-edd2f82becbe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ed694a74b8e540768ee5458fcb5ce649'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:36:54.088064 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1019/2059] 10.222.0.30 () {58 vars in 1224 bytes} [Tue Sep 6 04:36:53 2022] DELETE /identity/v3/users/ed694a74b8e540768ee5458fcb5ce649 => generated 0 bytes in 462 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 04:36:54.092649 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ddd48b-a688-42d8-881d-fa5d79ed389c None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:36:54.092937 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ddd48b-a688-42d8-881d-fa5d79ed389c None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:36:54.093213 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ddd48b-a688-42d8-881d-fa5d79ed389c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:36:54.112970 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-94ddd48b-a688-42d8-881d-fa5d79ed389c None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:36:54.114259 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-94ddd48b-a688-42d8-881d-fa5d79ed389c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:36:54.180428 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1041/2060] 10.222.0.30 () {58 vars in 958 bytes} [Tue Sep 6 04:36:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 04:36:54.191322 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6344f49-6843-40bd-ad8d-edd2f82becbe admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:36:54.192857 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-c6344f49-6843-40bd-ad8d-edd2f82becbe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:36:54.257743 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6344f49-6843-40bd-ad8d-edd2f82becbe admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:36:54.261249 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1e11a4-8678-4fc8-8630-b73955f30d8c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90492440f6a6471e8392361c098428aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b707733ce65c42edbc1fd95ad0a1c06c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:36:54.261938 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1e11a4-8678-4fc8-8630-b73955f30d8c service neutron] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:36:54.262219 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1e11a4-8678-4fc8-8630-b73955f30d8c service neutron] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:36:54.262496 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1e11a4-8678-4fc8-8630-b73955f30d8c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:36:54.272685 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e1e11a4-8678-4fc8-8630-b73955f30d8c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:36:54.274228 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e1e11a4-8678-4fc8-8630-b73955f30d8c service neutron] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:36:54.284461 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1020/2061] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Sep 6 04:36:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 04:36:54.895546 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ddd48b-a688-42d8-881d-fa5d79ed389c None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:36:54.897713 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-94ddd48b-a688-42d8-881d-fa5d79ed389c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:36:54.913405 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ddd48b-a688-42d8-881d-fa5d79ed389c None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:36:54.918295 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56540f01-1954-4dee-9cf1-eb5246ff440d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:36:54.920489 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56540f01-1954-4dee-9cf1-eb5246ff440d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:36:54.920489 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56540f01-1954-4dee-9cf1-eb5246ff440d admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:36:54.920489 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56540f01-1954-4dee-9cf1-eb5246ff440d admin admin] PATH_INFO: `/v3/projects/ea9efab8451a40919ea4206cc0a506da` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:36:54.921042 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56540f01-1954-4dee-9cf1-eb5246ff440d admin admin] RBAC: Authorizing `identity:delete_project(project_id=ea9efab8451a40919ea4206cc0a506da)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:36:54.922583 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56540f01-1954-4dee-9cf1-eb5246ff440d admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:36:55.048565 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-56540f01-1954-4dee-9cf1-eb5246ff440d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ea9efab8451a40919ea4206cc0a506da', 'request_id': 'req-56540f01-1954-4dee-9cf1-eb5246ff440d'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:36:55.072318 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-56540f01-1954-4dee-9cf1-eb5246ff440d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ea9efab8451a40919ea4206cc0a506da', 'request_id': 'req-56540f01-1954-4dee-9cf1-eb5246ff440d'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:36:55.120721 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-56540f01-1954-4dee-9cf1-eb5246ff440d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ea9efab8451a40919ea4206cc0a506da', 'request_id': 'req-56540f01-1954-4dee-9cf1-eb5246ff440d'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:36:55.133454 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-56540f01-1954-4dee-9cf1-eb5246ff440d admin admin] The token cache is being invalidate because project ea9efab8451a40919ea4206cc0a506da was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81698) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 04:36:55.133753 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-56540f01-1954-4dee-9cf1-eb5246ff440d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:36:55.134396 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-56540f01-1954-4dee-9cf1-eb5246ff440d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:36:55.135871 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1042/2062] 10.222.0.30 () {58 vars in 1233 bytes} [Tue Sep 6 04:36:54 2022] DELETE /identity/v3/projects/ea9efab8451a40919ea4206cc0a506da => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 04:36:55.140768 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5709ef06-6657-4f76-b7c3-6ac2d97753ec None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:36:55.141057 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5709ef06-6657-4f76-b7c3-6ac2d97753ec None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:36:55.141355 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5709ef06-6657-4f76-b7c3-6ac2d97753ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:36:55.161019 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-5709ef06-6657-4f76-b7c3-6ac2d97753ec None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:36:55.162278 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-5709ef06-6657-4f76-b7c3-6ac2d97753ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:36:55.192662 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:36:55.222246 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1021/2063] 10.222.0.30 () {58 vars in 958 bytes} [Tue Sep 6 04:36:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 04:36:55.230321 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a74086-5b5e-4d17-b58d-db7de1637d0f None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:36:55.230553 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a74086-5b5e-4d17-b58d-db7de1637d0f None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:36:55.230830 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a74086-5b5e-4d17-b58d-db7de1637d0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:36:55.251220 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-92a74086-5b5e-4d17-b58d-db7de1637d0f None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:36:55.252456 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-92a74086-5b5e-4d17-b58d-db7de1637d0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:36:55.260673 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1043/2064] 10.222.0.30 () {58 vars in 958 bytes} [Tue Sep 6 04:36:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 04:36:55.264898 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5709ef06-6657-4f76-b7c3-6ac2d97753ec None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:36:55.275637 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5709ef06-6657-4f76-b7c3-6ac2d97753ec None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:36:55.281607 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce67c2da-a8dc-48ff-8e14-001990392d16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:36:55.282207 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce67c2da-a8dc-48ff-8e14-001990392d16 admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:36:55.282483 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce67c2da-a8dc-48ff-8e14-001990392d16 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:36:55.282752 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce67c2da-a8dc-48ff-8e14-001990392d16 admin admin] PATH_INFO: `/v3/domains` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:36:55.283394 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce67c2da-a8dc-48ff-8e14-001990392d16 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:36:55.283691 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce67c2da-a8dc-48ff-8e14-001990392d16 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:36:55.286907 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce67c2da-a8dc-48ff-8e14-001990392d16 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:36:55.299443 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1022/2065] 10.222.0.30 () {58 vars in 1153 bytes} [Tue Sep 6 04:36:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:36:55.303729 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a74086-5b5e-4d17-b58d-db7de1637d0f None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:36:55.313297 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a74086-5b5e-4d17-b58d-db7de1637d0f None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:36:55.318641 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488009fc-dd95-4d81-85cb-3a40eba7dde7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:36:55.319256 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488009fc-dd95-4d81-85cb-3a40eba7dde7 admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:36:55.319543 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488009fc-dd95-4d81-85cb-3a40eba7dde7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:36:55.319818 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488009fc-dd95-4d81-85cb-3a40eba7dde7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:36:55.320458 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-488009fc-dd95-4d81-85cb-3a40eba7dde7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:36:55.322445 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-488009fc-dd95-4d81-85cb-3a40eba7dde7 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:36:55.337812 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:36:55.407631 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1044/2066] 10.222.0.30 () {60 vars in 1153 bytes} [Tue Sep 6 04:36:55 2022] POST /identity/v3/projects => generated 377 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 04:36:55.408891 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce67c2da-a8dc-48ff-8e14-001990392d16 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:36:55.428167 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce67c2da-a8dc-48ff-8e14-001990392d16 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:36:55.500241 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6432e792-e860-4e2d-9782-645d5361e80b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:36:55.500945 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6432e792-e860-4e2d-9782-645d5361e80b admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:36:55.501527 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6432e792-e860-4e2d-9782-645d5361e80b admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:36:55.501698 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6432e792-e860-4e2d-9782-645d5361e80b admin admin] PATH_INFO: `/v3/users` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:36:55.502350 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6432e792-e860-4e2d-9782-645d5361e80b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:36:55.505096 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6432e792-e860-4e2d-9782-645d5361e80b admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:36:55.624769 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1023/2067] 10.222.0.30 () {60 vars in 1144 bytes} [Tue Sep 6 04:36:55 2022] POST /identity/v3/users => generated 289 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 04:36:55.630303 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488009fc-dd95-4d81-85cb-3a40eba7dde7 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:36:55.640955 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488009fc-dd95-4d81-85cb-3a40eba7dde7 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:36:55.646901 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5fb373-2983-43bf-8738-ec4ff594cd08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:36:55.647521 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5fb373-2983-43bf-8738-ec4ff594cd08 admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:36:55.647801 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5fb373-2983-43bf-8738-ec4ff594cd08 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:36:55.648076 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5fb373-2983-43bf-8738-ec4ff594cd08 admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:36:55.648759 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5fb373-2983-43bf-8738-ec4ff594cd08 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:36:55.650735 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5fb373-2983-43bf-8738-ec4ff594cd08 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:36:55.666745 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1045/2068] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Sep 6 04:36:55 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 04:36:55.667005 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6432e792-e860-4e2d-9782-645d5361e80b admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:36:55.678928 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6432e792-e860-4e2d-9782-645d5361e80b admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:36:55.683911 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e029b12-f263-426b-8bcb-b9279bb701b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:36:55.684551 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e029b12-f263-426b-8bcb-b9279bb701b1 admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:36:55.684861 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e029b12-f263-426b-8bcb-b9279bb701b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:36:55.685137 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e029b12-f263-426b-8bcb-b9279bb701b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:36:55.685818 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e029b12-f263-426b-8bcb-b9279bb701b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:36:55.687422 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e029b12-f263-426b-8bcb-b9279bb701b1 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:36:55.702518 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1024/2069] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Sep 6 04:36:55 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 06 04:36:55.707066 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5fb373-2983-43bf-8738-ec4ff594cd08 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:36:55.717826 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5fb373-2983-43bf-8738-ec4ff594cd08 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:36:55.722365 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9c207a-825a-4fe6-9d47-e33b54166c43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:36:55.723089 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9c207a-825a-4fe6-9d47-e33b54166c43 admin admin] REQUEST_METHOD: `PUT` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:36:55.723364 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9c207a-825a-4fe6-9d47-e33b54166c43 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:36:55.723637 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9c207a-825a-4fe6-9d47-e33b54166c43 admin admin] PATH_INFO: `/v3/projects/e37b1d9d7f084c31b1a7937665bfb48e/users/83b9a7b470f04e27a64d9cad114ed9a8/roles/29607b2de39f40d68bb6ceef64b8a732` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:36:55.738179 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9c207a-825a-4fe6-9d47-e33b54166c43 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e37b1d9d7f084c31b1a7937665bfb48e, user_id=83b9a7b470f04e27a64d9cad114ed9a8, role_id=29607b2de39f40d68bb6ceef64b8a732)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:36:55.741439 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9c207a-825a-4fe6-9d47-e33b54166c43 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:36:55.807610 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1046/2070] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Sep 6 04:36:55 2022] PUT /identity/v3/projects/e37b1d9d7f084c31b1a7937665bfb48e/users/83b9a7b470f04e27a64d9cad114ed9a8/roles/29607b2de39f40d68bb6ceef64b8a732 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 04:36:55.819093 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caba8c0b-4a2b-4dd4-82e8-122981669e0b None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:36:55.819380 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caba8c0b-4a2b-4dd4-82e8-122981669e0b None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:36:55.819700 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caba8c0b-4a2b-4dd4-82e8-122981669e0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:36:55.851076 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-caba8c0b-4a2b-4dd4-82e8-122981669e0b None None] MFA Rules not processed for user `83b9a7b470f04e27a64d9cad114ed9a8`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:36:55.852279 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-caba8c0b-4a2b-4dd4-82e8-122981669e0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:36:55.916413 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1025/2071] 10.222.0.30 () {58 vars in 958 bytes} [Tue Sep 6 04:36:55 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 04:36:55.928644 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9c207a-825a-4fe6-9d47-e33b54166c43 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:36:55.930371 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-dc9c207a-825a-4fe6-9d47-e33b54166c43 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:36:55.987766 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9c207a-825a-4fe6-9d47-e33b54166c43 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:36:55.992071 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49888de-8efb-4667-9866-157d1131725b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd09b5ab675634a2eb4b03755743f7d0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b707733ce65c42edbc1fd95ad0a1c06c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:36:55.992705 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49888de-8efb-4667-9866-157d1131725b service cinder] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:36:55.993010 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49888de-8efb-4667-9866-157d1131725b service cinder] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:36:55.993281 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49888de-8efb-4667-9866-157d1131725b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:36:56.002077 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49888de-8efb-4667-9866-157d1131725b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:36:56.003583 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49888de-8efb-4667-9866-157d1131725b service cinder] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:36:56.013260 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1047/2072] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Sep 6 04:36:55 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 04:37:05.913705 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=84,keystone:INSERT=3 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:37:06.009163 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:INSERT=2,keystone:SELECT=43 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:37:35.691503 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caba8c0b-4a2b-4dd4-82e8-122981669e0b None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:37:35.692873 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-caba8c0b-4a2b-4dd4-82e8-122981669e0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:37:35.764149 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caba8c0b-4a2b-4dd4-82e8-122981669e0b None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:37:35.768871 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad877f6-303b-4249-aac1-18d55757129a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:37:35.769785 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad877f6-303b-4249-aac1-18d55757129a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:37:35.770024 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad877f6-303b-4249-aac1-18d55757129a admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:37:35.770373 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad877f6-303b-4249-aac1-18d55757129a admin admin] PATH_INFO: `/v3/users/88fd2421b7cf4bf78cd6fbc209660e30` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:37:35.771673 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad877f6-303b-4249-aac1-18d55757129a admin admin] RBAC: Authorizing `identity:delete_user(user_id=88fd2421b7cf4bf78cd6fbc209660e30)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:37:35.773842 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad877f6-303b-4249-aac1-18d55757129a admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:37:35.779415 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49888de-8efb-4667-9866-157d1131725b service cinder] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:37:35.781740 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-a49888de-8efb-4667-9866-157d1131725b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:37:35.795985 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49888de-8efb-4667-9866-157d1131725b service cinder] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:37:35.802506 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e43015-c5ee-4eba-be6b-7af4c9b3217e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:37:35.816130 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e43015-c5ee-4eba-be6b-7af4c9b3217e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:37:35.816867 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e43015-c5ee-4eba-be6b-7af4c9b3217e admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:37:35.817323 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e43015-c5ee-4eba-be6b-7af4c9b3217e admin admin] PATH_INFO: `/v3/users/d14b04d1ae9748c89a46b2994625bcad` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:37:35.824309 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e43015-c5ee-4eba-be6b-7af4c9b3217e admin admin] RBAC: Authorizing `identity:delete_user(user_id=d14b04d1ae9748c89a46b2994625bcad)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:37:35.824309 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e43015-c5ee-4eba-be6b-7af4c9b3217e admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:37:35.953159 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-4ad877f6-303b-4249-aac1-18d55757129a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '88fd2421b7cf4bf78cd6fbc209660e30'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:37:36.022721 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-4ad877f6-303b-4249-aac1-18d55757129a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '88fd2421b7cf4bf78cd6fbc209660e30'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:37:36.023351 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-4ad877f6-303b-4249-aac1-18d55757129a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '88fd2421b7cf4bf78cd6fbc209660e30'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:37:36.027721 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-b5e43015-c5ee-4eba-be6b-7af4c9b3217e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd14b04d1ae9748c89a46b2994625bcad'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:37:36.034970 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-4ad877f6-303b-4249-aac1-18d55757129a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '88fd2421b7cf4bf78cd6fbc209660e30'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:37:36.051679 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-4ad877f6-303b-4249-aac1-18d55757129a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '88fd2421b7cf4bf78cd6fbc209660e30'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:37:36.061724 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1026/2073] 10.222.0.30 () {58 vars in 1224 bytes} [Tue Sep 6 04:37:35 2022] DELETE /identity/v3/users/88fd2421b7cf4bf78cd6fbc209660e30 => generated 0 bytes in 369 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 04:37:36.063267 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c756c834-b658-4004-8569-2d2d6e549269 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:37:36.064266 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c756c834-b658-4004-8569-2d2d6e549269 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:37:36.064266 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c756c834-b658-4004-8569-2d2d6e549269 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:37:36.086225 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-c756c834-b658-4004-8569-2d2d6e549269 None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:37:36.087460 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-c756c834-b658-4004-8569-2d2d6e549269 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:37:36.090952 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-b5e43015-c5ee-4eba-be6b-7af4c9b3217e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd14b04d1ae9748c89a46b2994625bcad'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:37:36.091628 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-b5e43015-c5ee-4eba-be6b-7af4c9b3217e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd14b04d1ae9748c89a46b2994625bcad'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:37:36.113637 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-b5e43015-c5ee-4eba-be6b-7af4c9b3217e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd14b04d1ae9748c89a46b2994625bcad'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:37:36.122640 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-b5e43015-c5ee-4eba-be6b-7af4c9b3217e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd14b04d1ae9748c89a46b2994625bcad'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:37:36.128975 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1048/2074] 10.222.0.30 () {58 vars in 1224 bytes} [Tue Sep 6 04:37:35 2022] DELETE /identity/v3/users/d14b04d1ae9748c89a46b2994625bcad => generated 0 bytes in 350 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 04:37:36.133177 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc23a1f-df33-4007-8db1-7b27075b19c2 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:37:36.133496 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc23a1f-df33-4007-8db1-7b27075b19c2 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:37:36.133775 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc23a1f-df33-4007-8db1-7b27075b19c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:37:36.155741 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-5cc23a1f-df33-4007-8db1-7b27075b19c2 None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:37:36.158288 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-5cc23a1f-df33-4007-8db1-7b27075b19c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:37:36.165902 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1027/2075] 10.222.0.30 () {58 vars in 958 bytes} [Tue Sep 6 04:37:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 04:37:36.180592 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c756c834-b658-4004-8569-2d2d6e549269 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:37:36.182023 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-c756c834-b658-4004-8569-2d2d6e549269 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:37:36.204874 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:37:36.223659 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1049/2076] 10.222.0.30 () {58 vars in 958 bytes} [Tue Sep 6 04:37:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 04:37:36.233273 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc23a1f-df33-4007-8db1-7b27075b19c2 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:37:36.234749 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-5cc23a1f-df33-4007-8db1-7b27075b19c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:37:36.253599 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c756c834-b658-4004-8569-2d2d6e549269 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:37:36.304742 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b48105-b106-4f07-92fd-fb68f78ff101 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90492440f6a6471e8392361c098428aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b707733ce65c42edbc1fd95ad0a1c06c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:37:36.305416 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b48105-b106-4f07-92fd-fb68f78ff101 service neutron] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:37:36.305697 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b48105-b106-4f07-92fd-fb68f78ff101 service neutron] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:37:36.305970 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b48105-b106-4f07-92fd-fb68f78ff101 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:37:36.307672 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc23a1f-df33-4007-8db1-7b27075b19c2 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:37:36.311408 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3e8f7d-7302-4dab-bf28-d1c88cb27cbf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90492440f6a6471e8392361c098428aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b707733ce65c42edbc1fd95ad0a1c06c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:37:36.315373 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3e8f7d-7302-4dab-bf28-d1c88cb27cbf service neutron] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:37:36.315373 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3e8f7d-7302-4dab-bf28-d1c88cb27cbf service neutron] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:37:36.315373 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3e8f7d-7302-4dab-bf28-d1c88cb27cbf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:37:36.316644 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b48105-b106-4f07-92fd-fb68f78ff101 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:37:36.318099 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b48105-b106-4f07-92fd-fb68f78ff101 service neutron] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:37:36.321898 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3e8f7d-7302-4dab-bf28-d1c88cb27cbf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:37:36.323946 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3e8f7d-7302-4dab-bf28-d1c88cb27cbf service neutron] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:37:36.332678 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1028/2077] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Sep 6 04:37:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 06 04:37:36.333936 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1050/2078] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Sep 6 04:37:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 06 04:37:36.839735 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b48105-b106-4f07-92fd-fb68f78ff101 service neutron] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:37:36.839735 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-f9b48105-b106-4f07-92fd-fb68f78ff101 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:37:36.849977 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b48105-b106-4f07-92fd-fb68f78ff101 service neutron] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:37:36.864352 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3e8f7d-7302-4dab-bf28-d1c88cb27cbf service neutron] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:37:36.866296 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9ce029-f04f-452e-8e4c-d6eca62238ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:37:36.866780 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-3a3e8f7d-7302-4dab-bf28-d1c88cb27cbf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:37:36.867349 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9ce029-f04f-452e-8e4c-d6eca62238ef admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:37:36.867738 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9ce029-f04f-452e-8e4c-d6eca62238ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:37:36.868681 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9ce029-f04f-452e-8e4c-d6eca62238ef admin admin] PATH_INFO: `/v3/projects/a43bf76a0ad84d77b41d0115f0729fc1` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:37:36.870655 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9ce029-f04f-452e-8e4c-d6eca62238ef admin admin] RBAC: Authorizing `identity:delete_project(project_id=a43bf76a0ad84d77b41d0115f0729fc1)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:37:36.877058 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9ce029-f04f-452e-8e4c-d6eca62238ef admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:37:36.888144 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3e8f7d-7302-4dab-bf28-d1c88cb27cbf service neutron] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:37:36.893007 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74f1712-6827-4332-b284-557cfc7108ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:37:36.893751 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74f1712-6827-4332-b284-557cfc7108ef admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:37:36.894094 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74f1712-6827-4332-b284-557cfc7108ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:37:36.894518 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74f1712-6827-4332-b284-557cfc7108ef admin admin] PATH_INFO: `/v3/projects/0f5ff30b7d70423a809d6c3701149255` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:37:36.896081 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74f1712-6827-4332-b284-557cfc7108ef admin admin] RBAC: Authorizing `identity:delete_project(project_id=0f5ff30b7d70423a809d6c3701149255)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:37:36.898214 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74f1712-6827-4332-b284-557cfc7108ef admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:37:36.911159 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:37:37.000681 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-1d9ce029-f04f-452e-8e4c-d6eca62238ef admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a43bf76a0ad84d77b41d0115f0729fc1', 'request_id': 'req-1d9ce029-f04f-452e-8e4c-d6eca62238ef'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:37:37.057655 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-a74f1712-6827-4332-b284-557cfc7108ef admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0f5ff30b7d70423a809d6c3701149255', 'request_id': 'req-a74f1712-6827-4332-b284-557cfc7108ef'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:37:37.063921 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-a74f1712-6827-4332-b284-557cfc7108ef admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0f5ff30b7d70423a809d6c3701149255', 'request_id': 'req-a74f1712-6827-4332-b284-557cfc7108ef'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:37:37.068424 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-1d9ce029-f04f-452e-8e4c-d6eca62238ef admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a43bf76a0ad84d77b41d0115f0729fc1', 'request_id': 'req-1d9ce029-f04f-452e-8e4c-d6eca62238ef'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:37:37.076226 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-1d9ce029-f04f-452e-8e4c-d6eca62238ef admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a43bf76a0ad84d77b41d0115f0729fc1', 'request_id': 'req-1d9ce029-f04f-452e-8e4c-d6eca62238ef'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:37:37.088603 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-1d9ce029-f04f-452e-8e4c-d6eca62238ef admin admin] The token cache is being invalidate because project a43bf76a0ad84d77b41d0115f0729fc1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81697) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 04:37:37.090556 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-1d9ce029-f04f-452e-8e4c-d6eca62238ef admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:37:37.091309 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-1d9ce029-f04f-452e-8e4c-d6eca62238ef admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:37:37.093284 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1029/2079] 10.222.0.30 () {58 vars in 1233 bytes} [Tue Sep 6 04:37:36 2022] DELETE /identity/v3/projects/a43bf76a0ad84d77b41d0115f0729fc1 => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 04:37:37.098009 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8641ec64-1b5a-425c-b4e2-105cf806da3b None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:37:37.098407 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8641ec64-1b5a-425c-b4e2-105cf806da3b None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:37:37.098817 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8641ec64-1b5a-425c-b4e2-105cf806da3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:37:37.121816 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-8641ec64-1b5a-425c-b4e2-105cf806da3b None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:37:37.123182 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-8641ec64-1b5a-425c-b4e2-105cf806da3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:37:37.137937 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-a74f1712-6827-4332-b284-557cfc7108ef admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0f5ff30b7d70423a809d6c3701149255', 'request_id': 'req-a74f1712-6827-4332-b284-557cfc7108ef'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:37:37.154931 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-a74f1712-6827-4332-b284-557cfc7108ef admin admin] The token cache is being invalidate because project 0f5ff30b7d70423a809d6c3701149255 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81698) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 04:37:37.155443 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-a74f1712-6827-4332-b284-557cfc7108ef admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:37:37.156290 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-a74f1712-6827-4332-b284-557cfc7108ef admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:37:37.158171 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1051/2080] 10.222.0.30 () {58 vars in 1233 bytes} [Tue Sep 6 04:37:36 2022] DELETE /identity/v3/projects/0f5ff30b7d70423a809d6c3701149255 => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 04:37:37.163583 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1023d406-f4b4-469b-a5d6-d1678ff1cbca None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:37:37.164009 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1023d406-f4b4-469b-a5d6-d1678ff1cbca None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:37:37.164561 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1023d406-f4b4-469b-a5d6-d1678ff1cbca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:37:37.210216 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-1023d406-f4b4-469b-a5d6-d1678ff1cbca None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:37:37.211610 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-1023d406-f4b4-469b-a5d6-d1678ff1cbca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:37:37.222168 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1052/2081] 10.222.0.30 () {58 vars in 958 bytes} [Tue Sep 6 04:37:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 04:37:37.223026 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1030/2082] 10.222.0.30 () {58 vars in 958 bytes} [Tue Sep 6 04:37:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 04:37:37.224130 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebd7800-cd33-45fb-b173-0487082327ad None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:37:37.224414 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebd7800-cd33-45fb-b173-0487082327ad None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:37:37.224677 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebd7800-cd33-45fb-b173-0487082327ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:37:37.230096 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9865452c-0c50-467d-9736-eda35c38a141 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:37:37.230426 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9865452c-0c50-467d-9736-eda35c38a141 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:37:37.230797 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9865452c-0c50-467d-9736-eda35c38a141 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:37:37.241962 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:37:37.251092 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-6ebd7800-cd33-45fb-b173-0487082327ad None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:37:37.255308 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-6ebd7800-cd33-45fb-b173-0487082327ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:37:37.257855 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-9865452c-0c50-467d-9736-eda35c38a141 None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:37:37.259121 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-9865452c-0c50-467d-9736-eda35c38a141 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:37:37.267390 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1053/2083] 10.222.0.30 () {58 vars in 958 bytes} [Tue Sep 6 04:37:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 04:37:37.271500 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ebd7800-cd33-45fb-b173-0487082327ad None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:37:37.273662 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1031/2084] 10.222.0.30 () {58 vars in 958 bytes} [Tue Sep 6 04:37:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 04:37:37.285856 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9865452c-0c50-467d-9736-eda35c38a141 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:37:37.298802 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9865452c-0c50-467d-9736-eda35c38a141 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:37:37.300224 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ebd7800-cd33-45fb-b173-0487082327ad None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:37:37.304858 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d376f2d1-0402-48a3-bf12-17a0c1090d21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:37:37.305615 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d376f2d1-0402-48a3-bf12-17a0c1090d21 admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:37:37.305967 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d376f2d1-0402-48a3-bf12-17a0c1090d21 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:37:37.306509 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d376f2d1-0402-48a3-bf12-17a0c1090d21 admin admin] PATH_INFO: `/v3/domains` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:37:37.307146 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d376f2d1-0402-48a3-bf12-17a0c1090d21 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:37:37.307275 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7caac9-adf8-478d-9c37-22be7ab92fda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:37:37.308386 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d376f2d1-0402-48a3-bf12-17a0c1090d21 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:37:37.308924 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7caac9-adf8-478d-9c37-22be7ab92fda admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:37:37.309733 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7caac9-adf8-478d-9c37-22be7ab92fda admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:37:37.310192 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7caac9-adf8-478d-9c37-22be7ab92fda admin admin] PATH_INFO: `/v3/domains` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:37:37.310462 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d376f2d1-0402-48a3-bf12-17a0c1090d21 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:37:37.312137 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7caac9-adf8-478d-9c37-22be7ab92fda admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:37:37.312954 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7caac9-adf8-478d-9c37-22be7ab92fda admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:37:37.318173 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7caac9-adf8-478d-9c37-22be7ab92fda admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:37:37.327871 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1032/2085] 10.222.0.30 () {58 vars in 1153 bytes} [Tue Sep 6 04:37:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:37:37.328891 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d376f2d1-0402-48a3-bf12-17a0c1090d21 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:37:37.334373 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1054/2086] 10.222.0.30 () {58 vars in 1153 bytes} [Tue Sep 6 04:37:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 04:37:37.341051 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d376f2d1-0402-48a3-bf12-17a0c1090d21 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:37:37.341472 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7caac9-adf8-478d-9c37-22be7ab92fda admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:37:37.346516 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f56c5f-0796-44f4-94e5-d602df70bcef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:37:37.357636 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f56c5f-0796-44f4-94e5-d602df70bcef admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:37:37.357636 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f56c5f-0796-44f4-94e5-d602df70bcef admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:37:37.357636 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f56c5f-0796-44f4-94e5-d602df70bcef admin admin] PATH_INFO: `/v3/projects` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:37:37.357636 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f56c5f-0796-44f4-94e5-d602df70bcef admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:37:37.357636 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f56c5f-0796-44f4-94e5-d602df70bcef admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:37:37.358260 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7caac9-adf8-478d-9c37-22be7ab92fda admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:37:37.372211 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4fafca-252a-4169-aef9-514bb28ac036 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:37:37.372211 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4fafca-252a-4169-aef9-514bb28ac036 admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:37:37.372211 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4fafca-252a-4169-aef9-514bb28ac036 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:37:37.372211 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4fafca-252a-4169-aef9-514bb28ac036 admin admin] PATH_INFO: `/v3/projects` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:37:37.372972 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4fafca-252a-4169-aef9-514bb28ac036 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:37:37.375927 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4fafca-252a-4169-aef9-514bb28ac036 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:37:37.457823 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1033/2088] 10.222.0.30 () {60 vars in 1153 bytes} [Tue Sep 6 04:37:37 2022] POST /identity/v3/projects => generated 371 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 04:37:37.458054 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1055/2088] 10.222.0.30 () {60 vars in 1153 bytes} [Tue Sep 6 04:37:37 2022] POST /identity/v3/projects => generated 379 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 04:37:37.462087 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f56c5f-0796-44f4-94e5-d602df70bcef admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:37:37.462240 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4fafca-252a-4169-aef9-514bb28ac036 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:37:37.524508 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f56c5f-0796-44f4-94e5-d602df70bcef admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:37:37.526849 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4fafca-252a-4169-aef9-514bb28ac036 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:37:37.532094 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f36144-d7f4-4085-9f37-4af2676eddde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:37:37.532749 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f36144-d7f4-4085-9f37-4af2676eddde admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:37:37.533037 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f36144-d7f4-4085-9f37-4af2676eddde admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:37:37.533339 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f36144-d7f4-4085-9f37-4af2676eddde admin admin] PATH_INFO: `/v3/users` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:37:37.533459 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818c4a31-9305-432b-bce4-b971fb48fa34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:37:37.535085 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f36144-d7f4-4085-9f37-4af2676eddde admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:37:37.535991 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f36144-d7f4-4085-9f37-4af2676eddde admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:37:37.539837 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818c4a31-9305-432b-bce4-b971fb48fa34 admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:37:37.540584 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818c4a31-9305-432b-bce4-b971fb48fa34 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:37:37.540911 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818c4a31-9305-432b-bce4-b971fb48fa34 admin admin] PATH_INFO: `/v3/users` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:37:37.541610 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-818c4a31-9305-432b-bce4-b971fb48fa34 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:37:37.544306 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-818c4a31-9305-432b-bce4-b971fb48fa34 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:37:37.628630 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1034/2089] 10.222.0.30 () {60 vars in 1144 bytes} [Tue Sep 6 04:37:37 2022] POST /identity/v3/users => generated 290 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 04:37:37.631827 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1056/2090] 10.222.0.30 () {60 vars in 1144 bytes} [Tue Sep 6 04:37:37 2022] POST /identity/v3/users => generated 286 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 04:37:37.632672 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f36144-d7f4-4085-9f37-4af2676eddde admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:37:37.632806 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818c4a31-9305-432b-bce4-b971fb48fa34 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:37:37.643178 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f36144-d7f4-4085-9f37-4af2676eddde admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:37:37.643581 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818c4a31-9305-432b-bce4-b971fb48fa34 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:37:37.648161 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d64aadb-395f-48be-bba0-2eeedd3e0c33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:37:37.648300 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2047b1ba-f360-46e8-beb1-afc46e6b5f7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:37:37.648757 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d64aadb-395f-48be-bba0-2eeedd3e0c33 admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:37:37.648871 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2047b1ba-f360-46e8-beb1-afc46e6b5f7c admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:37:37.649147 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d64aadb-395f-48be-bba0-2eeedd3e0c33 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:37:37.649426 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2047b1ba-f360-46e8-beb1-afc46e6b5f7c admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:37:37.649525 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d64aadb-395f-48be-bba0-2eeedd3e0c33 admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:37:37.649709 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2047b1ba-f360-46e8-beb1-afc46e6b5f7c admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:37:37.650454 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d64aadb-395f-48be-bba0-2eeedd3e0c33 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:37:37.650614 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2047b1ba-f360-46e8-beb1-afc46e6b5f7c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:37:37.653031 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2047b1ba-f360-46e8-beb1-afc46e6b5f7c admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:37:37.653411 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d64aadb-395f-48be-bba0-2eeedd3e0c33 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:37:37.663965 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1057/2091] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Sep 6 04:37:37 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 04:37:37.664119 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1035/2092] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Sep 6 04:37:37 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 04:37:37.667931 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2047b1ba-f360-46e8-beb1-afc46e6b5f7c admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:37:37.670977 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d64aadb-395f-48be-bba0-2eeedd3e0c33 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:37:37.678040 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2047b1ba-f360-46e8-beb1-afc46e6b5f7c admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:37:37.681036 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d64aadb-395f-48be-bba0-2eeedd3e0c33 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:37:37.683226 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4957f648-a505-4203-961b-a196e3d69a05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:37:37.683591 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4957f648-a505-4203-961b-a196e3d69a05 admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:37:37.683843 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4957f648-a505-4203-961b-a196e3d69a05 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:37:37.684115 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4957f648-a505-4203-961b-a196e3d69a05 admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:37:37.684750 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4957f648-a505-4203-961b-a196e3d69a05 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:37:37.686381 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4957f648-a505-4203-961b-a196e3d69a05 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:37:37.687322 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789e1226-f41d-4fd8-b768-0563e28deac5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:37:37.687933 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789e1226-f41d-4fd8-b768-0563e28deac5 admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:37:37.688215 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789e1226-f41d-4fd8-b768-0563e28deac5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:37:37.688491 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789e1226-f41d-4fd8-b768-0563e28deac5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:37:37.689273 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789e1226-f41d-4fd8-b768-0563e28deac5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:37:37.690897 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789e1226-f41d-4fd8-b768-0563e28deac5 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:37:37.698612 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1058/2093] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Sep 6 04:37:37 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 04:37:37.702548 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4957f648-a505-4203-961b-a196e3d69a05 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:37:37.702931 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1036/2094] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Sep 6 04:37:37 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 06 04:37:37.706760 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789e1226-f41d-4fd8-b768-0563e28deac5 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:37:37.712568 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4957f648-a505-4203-961b-a196e3d69a05 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:37:37.717986 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789e1226-f41d-4fd8-b768-0563e28deac5 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:37:37.718131 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b25dfc5-6ebb-4347-8bee-e21714805c89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:37:37.718740 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b25dfc5-6ebb-4347-8bee-e21714805c89 admin admin] REQUEST_METHOD: `PUT` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:37:37.719019 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b25dfc5-6ebb-4347-8bee-e21714805c89 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:37:37.719289 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b25dfc5-6ebb-4347-8bee-e21714805c89 admin admin] PATH_INFO: `/v3/projects/375efd2890fc4b71af13439813f00ab0/users/4b6c761af9c5457ea1e481ce400aded8/roles/29607b2de39f40d68bb6ceef64b8a732` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:37:37.723878 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044197b9-f46c-4483-bf33-5d4f1bac42ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:37:37.724743 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044197b9-f46c-4483-bf33-5d4f1bac42ef admin admin] REQUEST_METHOD: `PUT` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:37:37.725167 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044197b9-f46c-4483-bf33-5d4f1bac42ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:37:37.725721 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044197b9-f46c-4483-bf33-5d4f1bac42ef admin admin] PATH_INFO: `/v3/projects/049dc6c356f9468db1e4c47ffd160189/users/2549d8df8b9e407383bc2e37e026cb6f/roles/29607b2de39f40d68bb6ceef64b8a732` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:37:37.735160 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b25dfc5-6ebb-4347-8bee-e21714805c89 admin admin] RBAC: Authorizing `identity:create_grant(project_id=375efd2890fc4b71af13439813f00ab0, user_id=4b6c761af9c5457ea1e481ce400aded8, role_id=29607b2de39f40d68bb6ceef64b8a732)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:37:37.744114 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b25dfc5-6ebb-4347-8bee-e21714805c89 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:37:37.744114 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:37:37.749482 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044197b9-f46c-4483-bf33-5d4f1bac42ef admin admin] RBAC: Authorizing `identity:create_grant(project_id=049dc6c356f9468db1e4c47ffd160189, user_id=2549d8df8b9e407383bc2e37e026cb6f, role_id=29607b2de39f40d68bb6ceef64b8a732)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:37:37.752329 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044197b9-f46c-4483-bf33-5d4f1bac42ef admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:37:37.811668 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1059/2095] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Sep 6 04:37:37 2022] PUT /identity/v3/projects/375efd2890fc4b71af13439813f00ab0/users/4b6c761af9c5457ea1e481ce400aded8/roles/29607b2de39f40d68bb6ceef64b8a732 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 04:37:37.816707 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0318c3f2-1ab9-459e-af5d-76753f0ea822 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:37:37.817508 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0318c3f2-1ab9-459e-af5d-76753f0ea822 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:37:37.818491 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0318c3f2-1ab9-459e-af5d-76753f0ea822 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:37:37.822871 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1037/2096] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Sep 6 04:37:37 2022] PUT /identity/v3/projects/049dc6c356f9468db1e4c47ffd160189/users/2549d8df8b9e407383bc2e37e026cb6f/roles/29607b2de39f40d68bb6ceef64b8a732 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 04:37:37.833960 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce10c0e4-7eed-4a7c-8321-19fa83b37d18 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:37:37.834242 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce10c0e4-7eed-4a7c-8321-19fa83b37d18 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:37:37.834510 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce10c0e4-7eed-4a7c-8321-19fa83b37d18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:37:37.842805 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-0318c3f2-1ab9-459e-af5d-76753f0ea822 None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:37:37.844045 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-0318c3f2-1ab9-459e-af5d-76753f0ea822 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:37:37.866298 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-ce10c0e4-7eed-4a7c-8321-19fa83b37d18 None None] MFA Rules not processed for user `2549d8df8b9e407383bc2e37e026cb6f`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:37:37.867604 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-ce10c0e4-7eed-4a7c-8321-19fa83b37d18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:37:37.909297 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1060/2097] 10.222.0.30 () {58 vars in 958 bytes} [Tue Sep 6 04:37:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 04:37:37.919864 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0318c3f2-1ab9-459e-af5d-76753f0ea822 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:37:37.921425 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-0318c3f2-1ab9-459e-af5d-76753f0ea822 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:37:37.922859 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:37:37.934415 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1038/2098] 10.222.0.30 () {58 vars in 958 bytes} [Tue Sep 6 04:37:37 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 04:37:37.939341 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce10c0e4-7eed-4a7c-8321-19fa83b37d18 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:37:37.951813 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce10c0e4-7eed-4a7c-8321-19fa83b37d18 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:37:37.956539 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4f6210-0997-4fc9-b246-26782764e007 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:37:37.957147 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4f6210-0997-4fc9-b246-26782764e007 admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:37:37.957445 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4f6210-0997-4fc9-b246-26782764e007 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:37:37.959933 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4f6210-0997-4fc9-b246-26782764e007 admin admin] PATH_INFO: `/v3/projects` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:37:37.959933 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4f6210-0997-4fc9-b246-26782764e007 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:37:37.960590 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4f6210-0997-4fc9-b246-26782764e007 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:37:37.984021 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0318c3f2-1ab9-459e-af5d-76753f0ea822 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:37:37.987754 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917203d8-5829-4d33-850e-465e0b71ebee service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90492440f6a6471e8392361c098428aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b707733ce65c42edbc1fd95ad0a1c06c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:37:37.988362 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917203d8-5829-4d33-850e-465e0b71ebee service neutron] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:37:37.988747 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917203d8-5829-4d33-850e-465e0b71ebee service neutron] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:37:37.989028 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917203d8-5829-4d33-850e-465e0b71ebee service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:37:37.998857 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917203d8-5829-4d33-850e-465e0b71ebee service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:37:38.000427 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917203d8-5829-4d33-850e-465e0b71ebee service neutron] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:37:38.010026 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1061/2099] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Sep 6 04:37:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 06 04:37:38.052763 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1039/2100] 10.222.0.30 () {60 vars in 1153 bytes} [Tue Sep 6 04:37:37 2022] POST /identity/v3/projects => generated 373 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 04:37:38.057201 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917203d8-5829-4d33-850e-465e0b71ebee service neutron] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:37:38.067680 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917203d8-5829-4d33-850e-465e0b71ebee service neutron] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:37:38.125098 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06b8d7a-3241-4c32-88d7-2ab9ee02769b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:37:38.125827 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06b8d7a-3241-4c32-88d7-2ab9ee02769b admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:37:38.126005 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06b8d7a-3241-4c32-88d7-2ab9ee02769b admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:37:38.126275 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06b8d7a-3241-4c32-88d7-2ab9ee02769b admin admin] PATH_INFO: `/v3/users` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:37:38.126920 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06b8d7a-3241-4c32-88d7-2ab9ee02769b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:37:38.128839 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06b8d7a-3241-4c32-88d7-2ab9ee02769b admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:37:38.243062 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1062/2101] 10.222.0.30 () {60 vars in 1144 bytes} [Tue Sep 6 04:37:38 2022] POST /identity/v3/users => generated 293 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 04:37:38.244376 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4f6210-0997-4fc9-b246-26782764e007 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:37:38.255992 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4f6210-0997-4fc9-b246-26782764e007 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:37:38.261546 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58c07b4-9515-4e80-b609-af84608d0bd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:37:38.262320 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58c07b4-9515-4e80-b609-af84608d0bd8 admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:37:38.262624 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58c07b4-9515-4e80-b609-af84608d0bd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:37:38.262907 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58c07b4-9515-4e80-b609-af84608d0bd8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:37:38.263869 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58c07b4-9515-4e80-b609-af84608d0bd8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:37:38.265206 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58c07b4-9515-4e80-b609-af84608d0bd8 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:37:38.279184 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1040/2102] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Sep 6 04:37:38 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 04:37:38.280432 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06b8d7a-3241-4c32-88d7-2ab9ee02769b admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:37:38.290851 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06b8d7a-3241-4c32-88d7-2ab9ee02769b admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:37:38.295697 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27af8a6-e3cd-48be-a36d-5e9387547d98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:37:38.296461 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27af8a6-e3cd-48be-a36d-5e9387547d98 admin admin] REQUEST_METHOD: `PUT` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:37:38.296808 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27af8a6-e3cd-48be-a36d-5e9387547d98 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:37:38.297157 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27af8a6-e3cd-48be-a36d-5e9387547d98 admin admin] PATH_INFO: `/v3/projects/ce9cc9cdc7964f4d990200d147010cf6/users/7eaee29c3f144f379ffa0d154583d1fa/roles/ea4c814713b149108e488de855b025e4` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:37:38.313509 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27af8a6-e3cd-48be-a36d-5e9387547d98 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ce9cc9cdc7964f4d990200d147010cf6, user_id=7eaee29c3f144f379ffa0d154583d1fa, role_id=ea4c814713b149108e488de855b025e4)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:37:38.316258 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27af8a6-e3cd-48be-a36d-5e9387547d98 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:37:38.418515 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1063/2103] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Sep 6 04:37:38 2022] PUT /identity/v3/projects/ce9cc9cdc7964f4d990200d147010cf6/users/7eaee29c3f144f379ffa0d154583d1fa/roles/ea4c814713b149108e488de855b025e4 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 04:37:38.421350 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58c07b4-9515-4e80-b609-af84608d0bd8 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:37:38.431626 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58c07b4-9515-4e80-b609-af84608d0bd8 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:37:38.492858 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a1509f-e9d1-4272-8bd4-a8c4e3919f34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:37:38.493562 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a1509f-e9d1-4272-8bd4-a8c4e3919f34 admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:37:38.493877 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a1509f-e9d1-4272-8bd4-a8c4e3919f34 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:37:38.494120 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a1509f-e9d1-4272-8bd4-a8c4e3919f34 admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:37:38.494795 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a1509f-e9d1-4272-8bd4-a8c4e3919f34 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:37:38.496448 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a1509f-e9d1-4272-8bd4-a8c4e3919f34 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:37:38.511625 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1041/2104] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Sep 6 04:37:38 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 06 04:37:38.513794 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27af8a6-e3cd-48be-a36d-5e9387547d98 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:37:38.523144 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:INSERT=5,keystone:SELECT=95 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:37:38.528689 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27af8a6-e3cd-48be-a36d-5e9387547d98 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:37:38.534417 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3075a56d-5bf9-4981-a0ef-88362fbe65a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:37:38.535159 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3075a56d-5bf9-4981-a0ef-88362fbe65a6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:37:38.535363 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3075a56d-5bf9-4981-a0ef-88362fbe65a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:37:38.535633 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3075a56d-5bf9-4981-a0ef-88362fbe65a6 admin admin] PATH_INFO: `/v3/projects/ce9cc9cdc7964f4d990200d147010cf6/users/7eaee29c3f144f379ffa0d154583d1fa/roles/ea4c814713b149108e488de855b025e4` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:37:38.537716 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3075a56d-5bf9-4981-a0ef-88362fbe65a6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ce9cc9cdc7964f4d990200d147010cf6, user_id=7eaee29c3f144f379ffa0d154583d1fa, role_id=ea4c814713b149108e488de855b025e4)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:37:38.540558 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3075a56d-5bf9-4981-a0ef-88362fbe65a6 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:37:38.551690 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1064/2105] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Sep 6 04:37:38 2022] PUT /identity/v3/projects/ce9cc9cdc7964f4d990200d147010cf6/users/7eaee29c3f144f379ffa0d154583d1fa/roles/ea4c814713b149108e488de855b025e4 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 04:37:38.563334 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b23960a-5701-48f2-bf35-b253e393663b None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:37:38.563677 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b23960a-5701-48f2-bf35-b253e393663b None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:37:38.564046 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b23960a-5701-48f2-bf35-b253e393663b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:37:38.597363 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-2b23960a-5701-48f2-bf35-b253e393663b None None] MFA Rules not processed for user `7eaee29c3f144f379ffa0d154583d1fa`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:37:38.598779 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-2b23960a-5701-48f2-bf35-b253e393663b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:37:38.677582 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1042/2106] 10.222.0.30 () {58 vars in 958 bytes} [Tue Sep 6 04:37:38 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 04:37:38.689215 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3075a56d-5bf9-4981-a0ef-88362fbe65a6 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:37:38.691102 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-3075a56d-5bf9-4981-a0ef-88362fbe65a6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:37:38.751550 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3075a56d-5bf9-4981-a0ef-88362fbe65a6 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:37:38.755802 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ff80b7-8867-4b5b-911a-d988211e247e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd09b5ab675634a2eb4b03755743f7d0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b707733ce65c42edbc1fd95ad0a1c06c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:37:38.756565 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ff80b7-8867-4b5b-911a-d988211e247e service cinder] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:37:38.756942 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ff80b7-8867-4b5b-911a-d988211e247e service cinder] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:37:38.757394 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ff80b7-8867-4b5b-911a-d988211e247e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:37:38.768271 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ff80b7-8867-4b5b-911a-d988211e247e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:37:38.773483 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ff80b7-8867-4b5b-911a-d988211e247e service cinder] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:37:38.780854 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1065/2107] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Sep 6 04:37:38 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 04:37:39.802188 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b23960a-5701-48f2-bf35-b253e393663b None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:37:39.867586 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:37:39.888990 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b23960a-5701-48f2-bf35-b253e393663b None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:37:39.897124 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4981ea08-159d-4d2d-881c-73c93057ae15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:37:39.897124 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4981ea08-159d-4d2d-881c-73c93057ae15 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:37:39.897124 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4981ea08-159d-4d2d-881c-73c93057ae15 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:37:39.897124 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4981ea08-159d-4d2d-881c-73c93057ae15 admin admin] PATH_INFO: `/v3/users/2549d8df8b9e407383bc2e37e026cb6f` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:37:39.897758 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4981ea08-159d-4d2d-881c-73c93057ae15 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2549d8df8b9e407383bc2e37e026cb6f)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:37:39.899289 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4981ea08-159d-4d2d-881c-73c93057ae15 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:37:40.096268 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-4981ea08-159d-4d2d-881c-73c93057ae15 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2549d8df8b9e407383bc2e37e026cb6f'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:37:40.205779 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-4981ea08-159d-4d2d-881c-73c93057ae15 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2549d8df8b9e407383bc2e37e026cb6f'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:37:40.206411 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-4981ea08-159d-4d2d-881c-73c93057ae15 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2549d8df8b9e407383bc2e37e026cb6f'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:37:40.261174 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-4981ea08-159d-4d2d-881c-73c93057ae15 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2549d8df8b9e407383bc2e37e026cb6f'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:37:40.268586 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-4981ea08-159d-4d2d-881c-73c93057ae15 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2549d8df8b9e407383bc2e37e026cb6f'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:37:40.300908 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1043/2108] 10.222.0.30 () {58 vars in 1224 bytes} [Tue Sep 6 04:37:39 2022] DELETE /identity/v3/users/2549d8df8b9e407383bc2e37e026cb6f => generated 0 bytes in 499 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 04:37:40.305390 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbd397e-e17b-483f-93ac-de4af3efe7e1 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:37:40.306813 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbd397e-e17b-483f-93ac-de4af3efe7e1 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:37:40.307286 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbd397e-e17b-483f-93ac-de4af3efe7e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:37:40.333847 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-4cbd397e-e17b-483f-93ac-de4af3efe7e1 None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:37:40.335087 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-4cbd397e-e17b-483f-93ac-de4af3efe7e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:37:40.419473 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1066/2109] 10.222.0.30 () {58 vars in 958 bytes} [Tue Sep 6 04:37:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 04:37:40.430404 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4981ea08-159d-4d2d-881c-73c93057ae15 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:37:40.431952 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-4981ea08-159d-4d2d-881c-73c93057ae15 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:37:40.491974 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4981ea08-159d-4d2d-881c-73c93057ae15 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:37:40.495763 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f0f13f-dd0f-4ac8-875f-34c365f17444 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90492440f6a6471e8392361c098428aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b707733ce65c42edbc1fd95ad0a1c06c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:37:40.496391 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f0f13f-dd0f-4ac8-875f-34c365f17444 service neutron] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:37:40.496663 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f0f13f-dd0f-4ac8-875f-34c365f17444 service neutron] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:37:40.496933 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f0f13f-dd0f-4ac8-875f-34c365f17444 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:37:40.532840 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f0f13f-dd0f-4ac8-875f-34c365f17444 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:37:40.532840 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f0f13f-dd0f-4ac8-875f-34c365f17444 service neutron] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:37:40.545854 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1044/2110] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Sep 6 04:37:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 04:37:41.184579 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cbd397e-e17b-483f-93ac-de4af3efe7e1 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:37:41.188477 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-4cbd397e-e17b-483f-93ac-de4af3efe7e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:37:41.200173 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cbd397e-e17b-483f-93ac-de4af3efe7e1 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:37:41.205882 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e545273-700b-4c06-99f7-3359d7201f0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:37:41.220155 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e545273-700b-4c06-99f7-3359d7201f0e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:37:41.220155 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e545273-700b-4c06-99f7-3359d7201f0e admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:37:41.220155 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e545273-700b-4c06-99f7-3359d7201f0e admin admin] PATH_INFO: `/v3/projects/049dc6c356f9468db1e4c47ffd160189` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:37:41.220155 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e545273-700b-4c06-99f7-3359d7201f0e admin admin] RBAC: Authorizing `identity:delete_project(project_id=049dc6c356f9468db1e4c47ffd160189)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:37:41.220155 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e545273-700b-4c06-99f7-3359d7201f0e admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:37:41.353649 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-9e545273-700b-4c06-99f7-3359d7201f0e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '049dc6c356f9468db1e4c47ffd160189', 'request_id': 'req-9e545273-700b-4c06-99f7-3359d7201f0e'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:37:41.398964 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-9e545273-700b-4c06-99f7-3359d7201f0e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '049dc6c356f9468db1e4c47ffd160189', 'request_id': 'req-9e545273-700b-4c06-99f7-3359d7201f0e'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:37:41.445520 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-9e545273-700b-4c06-99f7-3359d7201f0e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '049dc6c356f9468db1e4c47ffd160189', 'request_id': 'req-9e545273-700b-4c06-99f7-3359d7201f0e'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:37:41.468679 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-9e545273-700b-4c06-99f7-3359d7201f0e admin admin] The token cache is being invalidate because project 049dc6c356f9468db1e4c47ffd160189 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81698) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 04:37:41.469668 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-9e545273-700b-4c06-99f7-3359d7201f0e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:37:41.470492 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-9e545273-700b-4c06-99f7-3359d7201f0e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:37:41.472961 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1067/2111] 10.222.0.30 () {58 vars in 1233 bytes} [Tue Sep 6 04:37:41 2022] DELETE /identity/v3/projects/049dc6c356f9468db1e4c47ffd160189 => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 04:37:41.476975 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f0f13f-dd0f-4ac8-875f-34c365f17444 service neutron] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:37:41.490643 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f0f13f-dd0f-4ac8-875f-34c365f17444 service neutron] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:37:41.568730 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9625e5c5-4522-4f35-a9ea-00e451870bba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:37:41.569585 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9625e5c5-4522-4f35-a9ea-00e451870bba admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:37:41.571135 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9625e5c5-4522-4f35-a9ea-00e451870bba admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:37:41.571425 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9625e5c5-4522-4f35-a9ea-00e451870bba admin admin] PATH_INFO: `/v3/users/7eaee29c3f144f379ffa0d154583d1fa` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:37:41.572904 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9625e5c5-4522-4f35-a9ea-00e451870bba admin admin] RBAC: Authorizing `identity:delete_user(user_id=7eaee29c3f144f379ffa0d154583d1fa)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:37:41.575819 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9625e5c5-4522-4f35-a9ea-00e451870bba admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:37:41.587818 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:37:41.780346 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-9625e5c5-4522-4f35-a9ea-00e451870bba admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7eaee29c3f144f379ffa0d154583d1fa'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:37:41.846837 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-9625e5c5-4522-4f35-a9ea-00e451870bba admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7eaee29c3f144f379ffa0d154583d1fa'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:37:41.847713 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-9625e5c5-4522-4f35-a9ea-00e451870bba admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7eaee29c3f144f379ffa0d154583d1fa'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:37:41.857657 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-9625e5c5-4522-4f35-a9ea-00e451870bba admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7eaee29c3f144f379ffa0d154583d1fa'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:37:41.875950 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-9625e5c5-4522-4f35-a9ea-00e451870bba admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7eaee29c3f144f379ffa0d154583d1fa'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:37:41.886925 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1045/2112] 10.222.0.30 () {58 vars in 1224 bytes} [Tue Sep 6 04:37:41 2022] DELETE /identity/v3/users/7eaee29c3f144f379ffa0d154583d1fa => generated 0 bytes in 410 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 04:37:42.579067 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e545273-700b-4c06-99f7-3359d7201f0e admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:37:42.580643 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-9e545273-700b-4c06-99f7-3359d7201f0e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:37:42.599816 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e545273-700b-4c06-99f7-3359d7201f0e admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:37:42.627419 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:37:42.675676 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ec8f53-be7e-4985-becc-6b3c913c209e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:37:42.676810 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ec8f53-be7e-4985-becc-6b3c913c209e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:37:42.676810 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ec8f53-be7e-4985-becc-6b3c913c209e admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:37:42.677133 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ec8f53-be7e-4985-becc-6b3c913c209e admin admin] PATH_INFO: `/v3/projects/ce9cc9cdc7964f4d990200d147010cf6` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:37:42.678330 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ec8f53-be7e-4985-becc-6b3c913c209e admin admin] RBAC: Authorizing `identity:delete_project(project_id=ce9cc9cdc7964f4d990200d147010cf6)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:37:42.680410 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ec8f53-be7e-4985-becc-6b3c913c209e admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:37:42.806139 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-d8ec8f53-be7e-4985-becc-6b3c913c209e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ce9cc9cdc7964f4d990200d147010cf6', 'request_id': 'req-d8ec8f53-be7e-4985-becc-6b3c913c209e'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:37:42.813077 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-d8ec8f53-be7e-4985-becc-6b3c913c209e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ce9cc9cdc7964f4d990200d147010cf6', 'request_id': 'req-d8ec8f53-be7e-4985-becc-6b3c913c209e'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:37:42.879858 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-d8ec8f53-be7e-4985-becc-6b3c913c209e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ce9cc9cdc7964f4d990200d147010cf6', 'request_id': 'req-d8ec8f53-be7e-4985-becc-6b3c913c209e'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:37:42.901287 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-d8ec8f53-be7e-4985-becc-6b3c913c209e admin admin] The token cache is being invalidate because project ce9cc9cdc7964f4d990200d147010cf6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81698) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 04:37:42.901832 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-d8ec8f53-be7e-4985-becc-6b3c913c209e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:37:42.906003 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-d8ec8f53-be7e-4985-becc-6b3c913c209e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:37:42.909731 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1068/2113] 10.222.0.30 () {58 vars in 1233 bytes} [Tue Sep 6 04:37:42 2022] DELETE /identity/v3/projects/ce9cc9cdc7964f4d990200d147010cf6 => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 04:37:42.914296 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0b3f79-84bc-4c3c-b669-8336a26b45b6 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:37:42.914770 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0b3f79-84bc-4c3c-b669-8336a26b45b6 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:37:42.915523 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0b3f79-84bc-4c3c-b669-8336a26b45b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:37:42.940839 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-ca0b3f79-84bc-4c3c-b669-8336a26b45b6 None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:37:42.942678 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-ca0b3f79-84bc-4c3c-b669-8336a26b45b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:37:43.030797 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1046/2114] 10.222.0.30 () {58 vars in 958 bytes} [Tue Sep 6 04:37:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 04:37:43.038811 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5623b2-4722-4db0-8eee-0f58d9a86318 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:37:43.039118 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5623b2-4722-4db0-8eee-0f58d9a86318 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:37:43.043142 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5623b2-4722-4db0-8eee-0f58d9a86318 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:37:43.063623 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-0a5623b2-4722-4db0-8eee-0f58d9a86318 None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:37:43.064859 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-0a5623b2-4722-4db0-8eee-0f58d9a86318 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:37:43.074378 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1069/2115] 10.222.0.30 () {58 vars in 958 bytes} [Tue Sep 6 04:37:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 04:37:43.081447 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0b3f79-84bc-4c3c-b669-8336a26b45b6 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:37:43.092200 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0b3f79-84bc-4c3c-b669-8336a26b45b6 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:37:43.103222 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69af87a0-4d3c-4ad5-820b-0395312e3bc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:37:43.103952 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69af87a0-4d3c-4ad5-820b-0395312e3bc8 admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:37:43.104319 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69af87a0-4d3c-4ad5-820b-0395312e3bc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:37:43.105147 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69af87a0-4d3c-4ad5-820b-0395312e3bc8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:37:43.105960 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69af87a0-4d3c-4ad5-820b-0395312e3bc8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:37:43.106332 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69af87a0-4d3c-4ad5-820b-0395312e3bc8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:37:43.108078 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69af87a0-4d3c-4ad5-820b-0395312e3bc8 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:37:43.123985 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1047/2116] 10.222.0.30 () {58 vars in 1153 bytes} [Tue Sep 6 04:37:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:37:43.128045 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5623b2-4722-4db0-8eee-0f58d9a86318 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:37:43.139073 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5623b2-4722-4db0-8eee-0f58d9a86318 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:37:43.145799 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6896ad-8b43-421d-ae32-66afde52e8c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:37:43.147383 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6896ad-8b43-421d-ae32-66afde52e8c6 admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:37:43.147383 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6896ad-8b43-421d-ae32-66afde52e8c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:37:43.147383 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6896ad-8b43-421d-ae32-66afde52e8c6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:37:43.147974 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd6896ad-8b43-421d-ae32-66afde52e8c6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:37:43.149960 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd6896ad-8b43-421d-ae32-66afde52e8c6 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:37:43.238275 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1070/2117] 10.222.0.30 () {60 vars in 1153 bytes} [Tue Sep 6 04:37:43 2022] POST /identity/v3/projects => generated 369 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 04:37:43.242466 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69af87a0-4d3c-4ad5-820b-0395312e3bc8 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:37:43.254650 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69af87a0-4d3c-4ad5-820b-0395312e3bc8 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:37:43.355362 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f492228f-233c-4dda-b18f-b50877acb4a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:37:43.356069 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f492228f-233c-4dda-b18f-b50877acb4a4 admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:37:43.356959 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f492228f-233c-4dda-b18f-b50877acb4a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:37:43.357246 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f492228f-233c-4dda-b18f-b50877acb4a4 admin admin] PATH_INFO: `/v3/users` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:37:43.357927 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f492228f-233c-4dda-b18f-b50877acb4a4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:37:43.359869 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f492228f-233c-4dda-b18f-b50877acb4a4 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:37:43.478618 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:37:43.493193 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1048/2118] 10.222.0.30 () {60 vars in 1144 bytes} [Tue Sep 6 04:37:43 2022] POST /identity/v3/users => generated 285 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 04:37:43.494348 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6896ad-8b43-421d-ae32-66afde52e8c6 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:37:43.516750 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6896ad-8b43-421d-ae32-66afde52e8c6 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:37:43.524477 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54eba978-a6a3-4d13-8181-8b44c25f79ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:37:43.525476 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54eba978-a6a3-4d13-8181-8b44c25f79ec admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:37:43.525974 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54eba978-a6a3-4d13-8181-8b44c25f79ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:37:43.526257 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54eba978-a6a3-4d13-8181-8b44c25f79ec admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:37:43.527817 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54eba978-a6a3-4d13-8181-8b44c25f79ec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:37:43.532438 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54eba978-a6a3-4d13-8181-8b44c25f79ec admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:37:43.549182 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1071/2119] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Sep 6 04:37:43 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 06 04:37:43.554522 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f492228f-233c-4dda-b18f-b50877acb4a4 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:37:43.570884 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f492228f-233c-4dda-b18f-b50877acb4a4 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:37:43.585627 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3049ad22-3e46-4c85-8a94-658598c56fc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:37:43.586386 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3049ad22-3e46-4c85-8a94-658598c56fc4 admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:37:43.586760 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3049ad22-3e46-4c85-8a94-658598c56fc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:37:43.587076 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3049ad22-3e46-4c85-8a94-658598c56fc4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:37:43.587834 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3049ad22-3e46-4c85-8a94-658598c56fc4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:37:43.591098 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3049ad22-3e46-4c85-8a94-658598c56fc4 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:37:43.609495 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1049/2120] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Sep 6 04:37:43 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 04:37:43.614159 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54eba978-a6a3-4d13-8181-8b44c25f79ec admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:37:43.629566 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54eba978-a6a3-4d13-8181-8b44c25f79ec admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:37:43.638655 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947cae78-9c6a-4af4-99fb-04a4eb3b7204 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:37:43.640977 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947cae78-9c6a-4af4-99fb-04a4eb3b7204 admin admin] REQUEST_METHOD: `PUT` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:37:43.641345 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947cae78-9c6a-4af4-99fb-04a4eb3b7204 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:37:43.641709 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947cae78-9c6a-4af4-99fb-04a4eb3b7204 admin admin] PATH_INFO: `/v3/projects/1dd7acf998d8416f85341d4adbb40ba1/users/ca34f0d1c08a49f6b76c46b720977d3d/roles/29607b2de39f40d68bb6ceef64b8a732` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:37:43.665503 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-947cae78-9c6a-4af4-99fb-04a4eb3b7204 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1dd7acf998d8416f85341d4adbb40ba1, user_id=ca34f0d1c08a49f6b76c46b720977d3d, role_id=29607b2de39f40d68bb6ceef64b8a732)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:37:43.669605 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-947cae78-9c6a-4af4-99fb-04a4eb3b7204 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:37:43.740457 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1072/2121] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Sep 6 04:37:43 2022] PUT /identity/v3/projects/1dd7acf998d8416f85341d4adbb40ba1/users/ca34f0d1c08a49f6b76c46b720977d3d/roles/29607b2de39f40d68bb6ceef64b8a732 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 04:37:43.752408 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd5e5e0-62e1-4d88-a44e-d1c1c0800d09 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:37:43.752983 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd5e5e0-62e1-4d88-a44e-d1c1c0800d09 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:37:43.753717 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd5e5e0-62e1-4d88-a44e-d1c1c0800d09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:37:43.800395 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-afd5e5e0-62e1-4d88-a44e-d1c1c0800d09 None None] MFA Rules not processed for user `ca34f0d1c08a49f6b76c46b720977d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:37:43.802496 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-afd5e5e0-62e1-4d88-a44e-d1c1c0800d09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:37:43.872875 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6405e83a-df11-4a80-a90f-559e4da92e21 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:37:43.873188 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6405e83a-df11-4a80-a90f-559e4da92e21 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:37:43.873401 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6405e83a-df11-4a80-a90f-559e4da92e21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:37:43.914362 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-6405e83a-df11-4a80-a90f-559e4da92e21 None None] MFA Rules not processed for user `4b6c761af9c5457ea1e481ce400aded8`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:37:43.916672 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-6405e83a-df11-4a80-a90f-559e4da92e21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:37:43.936431 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1050/2122] 10.222.0.30 () {58 vars in 958 bytes} [Tue Sep 6 04:37:43 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 04:37:43.940890 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd5e5e0-62e1-4d88-a44e-d1c1c0800d09 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:37:43.960167 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:37:44.021472 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1073/2123] 10.222.0.30 () {58 vars in 958 bytes} [Tue Sep 6 04:37:43 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 04:37:44.025881 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6405e83a-df11-4a80-a90f-559e4da92e21 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:37:44.031111 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-6405e83a-df11-4a80-a90f-559e4da92e21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:37:44.055538 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd5e5e0-62e1-4d88-a44e-d1c1c0800d09 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:37:44.063527 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2864eb-13e7-405d-91f5-ecd8fae028d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:37:44.064506 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2864eb-13e7-405d-91f5-ecd8fae028d6 admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:37:44.064913 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2864eb-13e7-405d-91f5-ecd8fae028d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:37:44.065419 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2864eb-13e7-405d-91f5-ecd8fae028d6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:37:44.066316 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb2864eb-13e7-405d-91f5-ecd8fae028d6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:37:44.069222 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb2864eb-13e7-405d-91f5-ecd8fae028d6 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:37:44.109437 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6405e83a-df11-4a80-a90f-559e4da92e21 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:37:44.114741 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4470dc65-083d-4736-afa3-347697245cc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:37:44.115394 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4470dc65-083d-4736-afa3-347697245cc7 admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:37:44.115705 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4470dc65-083d-4736-afa3-347697245cc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:37:44.115985 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4470dc65-083d-4736-afa3-347697245cc7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:37:44.116630 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4470dc65-083d-4736-afa3-347697245cc7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:37:44.118658 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4470dc65-083d-4736-afa3-347697245cc7 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:37:44.167973 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1051/2124] 10.222.0.30 () {60 vars in 1153 bytes} [Tue Sep 6 04:37:43 2022] POST /identity/v3/projects => generated 367 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 04:37:44.171951 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2864eb-13e7-405d-91f5-ecd8fae028d6 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:37:44.222678 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1074/2125] 10.222.0.30 () {60 vars in 1153 bytes} [Tue Sep 6 04:37:44 2022] POST /identity/v3/projects => generated 377 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 04:37:44.224900 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4470dc65-083d-4736-afa3-347697245cc7 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:37:44.244598 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2864eb-13e7-405d-91f5-ecd8fae028d6 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:37:44.255361 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:37:44.298851 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4470dc65-083d-4736-afa3-347697245cc7 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:37:44.307999 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96853734-8499-4a88-8d9d-1815451e933e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:37:44.308863 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96853734-8499-4a88-8d9d-1815451e933e admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:37:44.309554 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96853734-8499-4a88-8d9d-1815451e933e admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:37:44.310145 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96853734-8499-4a88-8d9d-1815451e933e admin admin] PATH_INFO: `/v3/users` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:37:44.310802 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96853734-8499-4a88-8d9d-1815451e933e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:37:44.312938 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96853734-8499-4a88-8d9d-1815451e933e admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:37:44.314886 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030c2012-e18f-4844-ba0f-f255c9c790c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:37:44.315782 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030c2012-e18f-4844-ba0f-f255c9c790c0 admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:37:44.316136 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030c2012-e18f-4844-ba0f-f255c9c790c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:37:44.316488 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030c2012-e18f-4844-ba0f-f255c9c790c0 admin admin] PATH_INFO: `/v3/users` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:37:44.317202 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030c2012-e18f-4844-ba0f-f255c9c790c0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:37:44.319295 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030c2012-e18f-4844-ba0f-f255c9c790c0 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:37:44.440798 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1075/2126] 10.222.0.30 () {60 vars in 1144 bytes} [Tue Sep 6 04:37:44 2022] POST /identity/v3/users => generated 295 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 04:37:44.440798 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96853734-8499-4a88-8d9d-1815451e933e admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:37:44.449291 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96853734-8499-4a88-8d9d-1815451e933e admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:37:44.454111 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766f4b55-1c2a-4a6c-ad44-704c2876f8d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:37:44.454716 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766f4b55-1c2a-4a6c-ad44-704c2876f8d2 admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:37:44.454987 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766f4b55-1c2a-4a6c-ad44-704c2876f8d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:37:44.455260 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766f4b55-1c2a-4a6c-ad44-704c2876f8d2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:37:44.455909 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-766f4b55-1c2a-4a6c-ad44-704c2876f8d2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:37:44.457557 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-766f4b55-1c2a-4a6c-ad44-704c2876f8d2 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:37:44.469153 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1076/2127] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Sep 6 04:37:44 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 06 04:37:44.469544 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1052/2128] 10.222.0.30 () {60 vars in 1144 bytes} [Tue Sep 6 04:37:44 2022] POST /identity/v3/users => generated 290 bytes in 298 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 04:37:44.477841 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766f4b55-1c2a-4a6c-ad44-704c2876f8d2 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:37:44.487738 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030c2012-e18f-4844-ba0f-f255c9c790c0 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:37:44.489479 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766f4b55-1c2a-4a6c-ad44-704c2876f8d2 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:37:44.494847 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a9030d-db29-4a12-ba22-76ed32f50c25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:37:44.496051 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a9030d-db29-4a12-ba22-76ed32f50c25 admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:37:44.496332 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a9030d-db29-4a12-ba22-76ed32f50c25 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:37:44.496635 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a9030d-db29-4a12-ba22-76ed32f50c25 admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:37:44.497511 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a9030d-db29-4a12-ba22-76ed32f50c25 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:37:44.499181 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a9030d-db29-4a12-ba22-76ed32f50c25 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:37:44.512718 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030c2012-e18f-4844-ba0f-f255c9c790c0 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:37:44.513639 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1077/2129] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Sep 6 04:37:44 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 04:37:44.515538 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a9030d-db29-4a12-ba22-76ed32f50c25 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:37:44.526594 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a9030d-db29-4a12-ba22-76ed32f50c25 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:37:44.531551 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9986fcdf-ee7c-4234-83bf-4912b3ba1589 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:37:44.532271 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9986fcdf-ee7c-4234-83bf-4912b3ba1589 admin admin] REQUEST_METHOD: `PUT` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:37:44.533051 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9986fcdf-ee7c-4234-83bf-4912b3ba1589 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:37:44.533051 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9986fcdf-ee7c-4234-83bf-4912b3ba1589 admin admin] PATH_INFO: `/v3/projects/835080f9d50741e89ab94156ddb151a2/users/3bc4eebf14be4ccdbef971a3c08a9528/roles/ea4c814713b149108e488de855b025e4` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:37:44.534534 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40f068f-594b-4459-9048-1eab98beb0e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:37:44.535644 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40f068f-594b-4459-9048-1eab98beb0e5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:37:44.535935 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40f068f-594b-4459-9048-1eab98beb0e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:37:44.536267 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40f068f-594b-4459-9048-1eab98beb0e5 admin admin] PATH_INFO: `/v3/projects/7191b1fbea424d14b85ea63448f414be/users/d8f4eb17e3a843c7abb537e13744be55/roles/ea4c814713b149108e488de855b025e4` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:37:44.560524 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9986fcdf-ee7c-4234-83bf-4912b3ba1589 admin admin] RBAC: Authorizing `identity:create_grant(project_id=835080f9d50741e89ab94156ddb151a2, user_id=3bc4eebf14be4ccdbef971a3c08a9528, role_id=ea4c814713b149108e488de855b025e4)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:37:44.562661 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40f068f-594b-4459-9048-1eab98beb0e5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7191b1fbea424d14b85ea63448f414be, user_id=d8f4eb17e3a843c7abb537e13744be55, role_id=ea4c814713b149108e488de855b025e4)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:37:44.563381 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9986fcdf-ee7c-4234-83bf-4912b3ba1589 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:37:44.566225 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40f068f-594b-4459-9048-1eab98beb0e5 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:37:44.624221 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:37:44.629431 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1078/2130] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Sep 6 04:37:44 2022] PUT /identity/v3/projects/835080f9d50741e89ab94156ddb151a2/users/3bc4eebf14be4ccdbef971a3c08a9528/roles/ea4c814713b149108e488de855b025e4 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 04:37:44.633419 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9986fcdf-ee7c-4234-83bf-4912b3ba1589 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:37:44.653126 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1053/2131] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Sep 6 04:37:44 2022] PUT /identity/v3/projects/7191b1fbea424d14b85ea63448f414be/users/d8f4eb17e3a843c7abb537e13744be55/roles/ea4c814713b149108e488de855b025e4 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 04:37:44.659105 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40f068f-594b-4459-9048-1eab98beb0e5 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:37:44.717562 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9986fcdf-ee7c-4234-83bf-4912b3ba1589 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:37:44.739731 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40f068f-594b-4459-9048-1eab98beb0e5 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:37:44.748247 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c8fcef-389e-40e7-beea-3aef200d78a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:37:44.748875 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c8fcef-389e-40e7-beea-3aef200d78a0 admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:37:44.749166 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c8fcef-389e-40e7-beea-3aef200d78a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:37:44.749480 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c8fcef-389e-40e7-beea-3aef200d78a0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:37:44.750136 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c8fcef-389e-40e7-beea-3aef200d78a0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:37:44.751745 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c8fcef-389e-40e7-beea-3aef200d78a0 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:37:44.762609 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1054/2132] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Sep 6 04:37:44 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 06 04:37:44.766737 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c8fcef-389e-40e7-beea-3aef200d78a0 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:37:44.778999 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c8fcef-389e-40e7-beea-3aef200d78a0 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:37:44.783809 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7fec60-ff57-4ed6-bfc2-8e0259970357 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:37:44.790485 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7fec60-ff57-4ed6-bfc2-8e0259970357 admin admin] REQUEST_METHOD: `PUT` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:37:44.790485 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7fec60-ff57-4ed6-bfc2-8e0259970357 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:37:44.790485 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7fec60-ff57-4ed6-bfc2-8e0259970357 admin admin] PATH_INFO: `/v3/projects/7191b1fbea424d14b85ea63448f414be/users/d8f4eb17e3a843c7abb537e13744be55/roles/ea4c814713b149108e488de855b025e4` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:37:44.790485 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed7fec60-ff57-4ed6-bfc2-8e0259970357 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7191b1fbea424d14b85ea63448f414be, user_id=d8f4eb17e3a843c7abb537e13744be55, role_id=ea4c814713b149108e488de855b025e4)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:37:44.790930 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a53d01-ca14-4899-9bd8-616b2417a96b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:37:44.790930 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a53d01-ca14-4899-9bd8-616b2417a96b admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:37:44.790930 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a53d01-ca14-4899-9bd8-616b2417a96b admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:37:44.790930 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a53d01-ca14-4899-9bd8-616b2417a96b admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:37:44.791270 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a53d01-ca14-4899-9bd8-616b2417a96b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:37:44.791366 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed7fec60-ff57-4ed6-bfc2-8e0259970357 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:37:44.791449 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a53d01-ca14-4899-9bd8-616b2417a96b admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:37:44.800641 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1055/2133] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Sep 6 04:37:44 2022] PUT /identity/v3/projects/7191b1fbea424d14b85ea63448f414be/users/d8f4eb17e3a843c7abb537e13744be55/roles/ea4c814713b149108e488de855b025e4 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 04:37:44.804124 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1079/2134] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Sep 6 04:37:44 2022] GET /identity/v3/roles => generated 1420 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 06 04:37:44.808218 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7fec60-ff57-4ed6-bfc2-8e0259970357 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:37:44.875151 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7fec60-ff57-4ed6-bfc2-8e0259970357 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:37:44.880523 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5f04bb-98bb-4f70-9c18-1e1b85575482 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:37:44.881272 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5f04bb-98bb-4f70-9c18-1e1b85575482 admin admin] REQUEST_METHOD: `PUT` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:37:44.881509 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5f04bb-98bb-4f70-9c18-1e1b85575482 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:37:44.881789 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5f04bb-98bb-4f70-9c18-1e1b85575482 admin admin] PATH_INFO: `/v3/projects/835080f9d50741e89ab94156ddb151a2/users/3bc4eebf14be4ccdbef971a3c08a9528/roles/ea4c814713b149108e488de855b025e4` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:37:44.883809 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5f04bb-98bb-4f70-9c18-1e1b85575482 admin admin] RBAC: Authorizing `identity:create_grant(project_id=835080f9d50741e89ab94156ddb151a2, user_id=3bc4eebf14be4ccdbef971a3c08a9528, role_id=ea4c814713b149108e488de855b025e4)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:37:44.886779 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5f04bb-98bb-4f70-9c18-1e1b85575482 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:37:44.890796 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:37:44.901878 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1056/2135] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Sep 6 04:37:44 2022] PUT /identity/v3/projects/835080f9d50741e89ab94156ddb151a2/users/3bc4eebf14be4ccdbef971a3c08a9528/roles/ea4c814713b149108e488de855b025e4 => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 04:37:44.913053 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f738469-aa46-4fd4-9546-c3ec7749494e None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:37:44.919285 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f738469-aa46-4fd4-9546-c3ec7749494e None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:37:44.919285 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f738469-aa46-4fd4-9546-c3ec7749494e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:37:44.945022 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-9f738469-aa46-4fd4-9546-c3ec7749494e None None] MFA Rules not processed for user `3bc4eebf14be4ccdbef971a3c08a9528`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:37:44.946369 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-9f738469-aa46-4fd4-9546-c3ec7749494e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:37:45.020632 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1080/2136] 10.222.0.30 () {58 vars in 958 bytes} [Tue Sep 6 04:37:44 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 04:37:45.032940 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5f04bb-98bb-4f70-9c18-1e1b85575482 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:37:45.034724 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-eb5f04bb-98bb-4f70-9c18-1e1b85575482 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:37:45.096084 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5f04bb-98bb-4f70-9c18-1e1b85575482 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:37:45.100304 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be884fbb-76e1-4b53-bffc-e6bba62bddb6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd09b5ab675634a2eb4b03755743f7d0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b707733ce65c42edbc1fd95ad0a1c06c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:37:45.100924 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be884fbb-76e1-4b53-bffc-e6bba62bddb6 service cinder] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:37:45.101200 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be884fbb-76e1-4b53-bffc-e6bba62bddb6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:37:45.101514 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be884fbb-76e1-4b53-bffc-e6bba62bddb6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:37:45.112347 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be884fbb-76e1-4b53-bffc-e6bba62bddb6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:37:45.113846 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be884fbb-76e1-4b53-bffc-e6bba62bddb6 service cinder] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:37:45.124526 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1057/2137] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Sep 6 04:37:45 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 04:37:48.732429 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f738469-aa46-4fd4-9546-c3ec7749494e None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:37:48.772466 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:37:48.811021 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f738469-aa46-4fd4-9546-c3ec7749494e None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:37:48.815728 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f69830c-0844-4a27-a0d8-2e13a248ef1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:37:48.816369 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f69830c-0844-4a27-a0d8-2e13a248ef1b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:37:48.816653 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f69830c-0844-4a27-a0d8-2e13a248ef1b admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:37:48.816934 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f69830c-0844-4a27-a0d8-2e13a248ef1b admin admin] PATH_INFO: `/v3/users/ca34f0d1c08a49f6b76c46b720977d3d` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:37:48.818137 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f69830c-0844-4a27-a0d8-2e13a248ef1b admin admin] RBAC: Authorizing `identity:delete_user(user_id=ca34f0d1c08a49f6b76c46b720977d3d)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:37:48.820121 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f69830c-0844-4a27-a0d8-2e13a248ef1b admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:37:49.032040 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-0f69830c-0844-4a27-a0d8-2e13a248ef1b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ca34f0d1c08a49f6b76c46b720977d3d'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:37:49.125596 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-0f69830c-0844-4a27-a0d8-2e13a248ef1b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ca34f0d1c08a49f6b76c46b720977d3d'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:37:49.126779 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-0f69830c-0844-4a27-a0d8-2e13a248ef1b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ca34f0d1c08a49f6b76c46b720977d3d'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:37:49.143726 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-0f69830c-0844-4a27-a0d8-2e13a248ef1b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ca34f0d1c08a49f6b76c46b720977d3d'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:37:49.152436 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-0f69830c-0844-4a27-a0d8-2e13a248ef1b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ca34f0d1c08a49f6b76c46b720977d3d'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:37:49.161901 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1081/2138] 10.222.0.30 () {58 vars in 1224 bytes} [Tue Sep 6 04:37:48 2022] DELETE /identity/v3/users/ca34f0d1c08a49f6b76c46b720977d3d => generated 0 bytes in 430 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 04:37:49.166536 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ceec910-13a1-4c85-930a-b70de1c228e3 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:37:49.167425 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ceec910-13a1-4c85-930a-b70de1c228e3 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:37:49.167772 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ceec910-13a1-4c85-930a-b70de1c228e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:37:49.188489 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-4ceec910-13a1-4c85-930a-b70de1c228e3 None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:37:49.190248 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-4ceec910-13a1-4c85-930a-b70de1c228e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:37:49.276709 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1058/2139] 10.222.0.30 () {58 vars in 958 bytes} [Tue Sep 6 04:37:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 04:37:49.288345 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f69830c-0844-4a27-a0d8-2e13a248ef1b admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:37:49.290038 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-0f69830c-0844-4a27-a0d8-2e13a248ef1b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:37:49.395835 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f69830c-0844-4a27-a0d8-2e13a248ef1b admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:37:49.400929 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbcc365b-2525-41fd-8f4f-5279dd5b39d3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90492440f6a6471e8392361c098428aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b707733ce65c42edbc1fd95ad0a1c06c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:37:49.401999 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcc365b-2525-41fd-8f4f-5279dd5b39d3 service neutron] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:37:49.402309 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcc365b-2525-41fd-8f4f-5279dd5b39d3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:37:49.402642 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcc365b-2525-41fd-8f4f-5279dd5b39d3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:37:49.414074 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbcc365b-2525-41fd-8f4f-5279dd5b39d3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:37:49.415556 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbcc365b-2525-41fd-8f4f-5279dd5b39d3 service neutron] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:37:49.441534 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1082/2140] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Sep 6 04:37:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 04:37:50.331777 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ceec910-13a1-4c85-930a-b70de1c228e3 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:37:50.333788 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-4ceec910-13a1-4c85-930a-b70de1c228e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:37:50.349668 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ceec910-13a1-4c85-930a-b70de1c228e3 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:37:50.357773 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688fdc73-7cb0-43c4-9423-23c26a368528 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:37:50.358774 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688fdc73-7cb0-43c4-9423-23c26a368528 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:37:50.359223 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688fdc73-7cb0-43c4-9423-23c26a368528 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:37:50.359703 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688fdc73-7cb0-43c4-9423-23c26a368528 admin admin] PATH_INFO: `/v3/projects/1dd7acf998d8416f85341d4adbb40ba1` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:37:50.447240 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688fdc73-7cb0-43c4-9423-23c26a368528 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1dd7acf998d8416f85341d4adbb40ba1)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:37:50.447240 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688fdc73-7cb0-43c4-9423-23c26a368528 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:37:50.840535 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-688fdc73-7cb0-43c4-9423-23c26a368528 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1dd7acf998d8416f85341d4adbb40ba1', 'request_id': 'req-688fdc73-7cb0-43c4-9423-23c26a368528'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:37:50.995043 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-688fdc73-7cb0-43c4-9423-23c26a368528 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1dd7acf998d8416f85341d4adbb40ba1', 'request_id': 'req-688fdc73-7cb0-43c4-9423-23c26a368528'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:37:51.004957 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-688fdc73-7cb0-43c4-9423-23c26a368528 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1dd7acf998d8416f85341d4adbb40ba1', 'request_id': 'req-688fdc73-7cb0-43c4-9423-23c26a368528'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:37:51.044175 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-688fdc73-7cb0-43c4-9423-23c26a368528 admin admin] The token cache is being invalidate because project 1dd7acf998d8416f85341d4adbb40ba1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81697) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 04:37:51.044479 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-688fdc73-7cb0-43c4-9423-23c26a368528 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:37:51.048215 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-688fdc73-7cb0-43c4-9423-23c26a368528 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:37:51.055055 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1059/2141] 10.222.0.30 () {58 vars in 1233 bytes} [Tue Sep 6 04:37:50 2022] DELETE /identity/v3/projects/1dd7acf998d8416f85341d4adbb40ba1 => generated 0 bytes in 721 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 04:37:51.055944 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbcc365b-2525-41fd-8f4f-5279dd5b39d3 service neutron] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:37:51.075691 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbcc365b-2525-41fd-8f4f-5279dd5b39d3 service neutron] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:37:51.168127 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:37:51.199972 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e763e5-c46c-4723-a094-13d87a22eec2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:37:51.200636 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e763e5-c46c-4723-a094-13d87a22eec2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:37:51.201018 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e763e5-c46c-4723-a094-13d87a22eec2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:37:51.201466 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e763e5-c46c-4723-a094-13d87a22eec2 admin admin] PATH_INFO: `/v3/users/3bc4eebf14be4ccdbef971a3c08a9528` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:37:51.207440 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e763e5-c46c-4723-a094-13d87a22eec2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3bc4eebf14be4ccdbef971a3c08a9528)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:37:51.212628 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e763e5-c46c-4723-a094-13d87a22eec2 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:37:51.326668 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf77e2ee-9c1f-42ff-a078-03654afb0a73 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:37:51.327027 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf77e2ee-9c1f-42ff-a078-03654afb0a73 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:37:51.330056 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf77e2ee-9c1f-42ff-a078-03654afb0a73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:37:51.392698 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-bf77e2ee-9c1f-42ff-a078-03654afb0a73 None None] MFA Rules not processed for user `d8f4eb17e3a843c7abb537e13744be55`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:37:51.394171 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-bf77e2ee-9c1f-42ff-a078-03654afb0a73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:37:51.419010 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:INSERT=2,keystone:SELECT=91,keystone:DELETE=7 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:37:51.463519 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-62e763e5-c46c-4723-a094-13d87a22eec2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3bc4eebf14be4ccdbef971a3c08a9528'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:37:51.560986 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1060/2142] 10.222.0.30 () {58 vars in 958 bytes} [Tue Sep 6 04:37:51 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 04:37:51.571809 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-62e763e5-c46c-4723-a094-13d87a22eec2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3bc4eebf14be4ccdbef971a3c08a9528'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:37:51.572454 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-62e763e5-c46c-4723-a094-13d87a22eec2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3bc4eebf14be4ccdbef971a3c08a9528'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:37:51.583992 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf77e2ee-9c1f-42ff-a078-03654afb0a73 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:37:51.590781 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-bf77e2ee-9c1f-42ff-a078-03654afb0a73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:37:51.601080 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-62e763e5-c46c-4723-a094-13d87a22eec2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3bc4eebf14be4ccdbef971a3c08a9528'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:37:51.609030 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-62e763e5-c46c-4723-a094-13d87a22eec2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3bc4eebf14be4ccdbef971a3c08a9528'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:37:51.616134 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1083/2143] 10.222.0.30 () {58 vars in 1224 bytes} [Tue Sep 6 04:37:51 2022] DELETE /identity/v3/users/3bc4eebf14be4ccdbef971a3c08a9528 => generated 0 bytes in 561 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 04:37:51.704595 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf77e2ee-9c1f-42ff-a078-03654afb0a73 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:37:51.780076 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d614b6b-44bf-4f89-923e-f30a3d1b745c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9db138b77afa4b068244b8ef715404fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b707733ce65c42edbc1fd95ad0a1c06c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:37:51.780658 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d614b6b-44bf-4f89-923e-f30a3d1b745c service nova] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:37:51.781118 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d614b6b-44bf-4f89-923e-f30a3d1b745c service nova] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:37:51.781386 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d614b6b-44bf-4f89-923e-f30a3d1b745c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:37:51.783166 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-1d614b6b-44bf-4f89-923e-f30a3d1b745c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:37:51.849072 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d614b6b-44bf-4f89-923e-f30a3d1b745c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:37:51.850913 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d614b6b-44bf-4f89-923e-f30a3d1b745c service nova] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:37:51.873127 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1061/2144] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Sep 6 04:37:51 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 04:37:52.091285 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e763e5-c46c-4723-a094-13d87a22eec2 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:37:52.091285 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-62e763e5-c46c-4723-a094-13d87a22eec2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:37:52.186435 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e763e5-c46c-4723-a094-13d87a22eec2 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:37:52.193883 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b68002a-2f07-45fd-9fb4-541d02a73489 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:37:52.195098 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b68002a-2f07-45fd-9fb4-541d02a73489 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:37:52.195444 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b68002a-2f07-45fd-9fb4-541d02a73489 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:37:52.195777 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b68002a-2f07-45fd-9fb4-541d02a73489 admin admin] PATH_INFO: `/v3/projects/835080f9d50741e89ab94156ddb151a2` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:37:52.196970 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b68002a-2f07-45fd-9fb4-541d02a73489 admin admin] RBAC: Authorizing `identity:delete_project(project_id=835080f9d50741e89ab94156ddb151a2)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:37:52.203288 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b68002a-2f07-45fd-9fb4-541d02a73489 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:37:52.373589 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-2b68002a-2f07-45fd-9fb4-541d02a73489 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '835080f9d50741e89ab94156ddb151a2', 'request_id': 'req-2b68002a-2f07-45fd-9fb4-541d02a73489'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:37:52.382869 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-2b68002a-2f07-45fd-9fb4-541d02a73489 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '835080f9d50741e89ab94156ddb151a2', 'request_id': 'req-2b68002a-2f07-45fd-9fb4-541d02a73489'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:37:52.451812 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-2b68002a-2f07-45fd-9fb4-541d02a73489 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '835080f9d50741e89ab94156ddb151a2', 'request_id': 'req-2b68002a-2f07-45fd-9fb4-541d02a73489'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:37:52.472967 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-2b68002a-2f07-45fd-9fb4-541d02a73489 admin admin] The token cache is being invalidate because project 835080f9d50741e89ab94156ddb151a2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81698) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 04:37:52.473437 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-2b68002a-2f07-45fd-9fb4-541d02a73489 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:37:52.474118 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-2b68002a-2f07-45fd-9fb4-541d02a73489 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:37:52.478832 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1084/2145] 10.222.0.30 () {58 vars in 1233 bytes} [Tue Sep 6 04:37:52 2022] DELETE /identity/v3/projects/835080f9d50741e89ab94156ddb151a2 => generated 0 bytes in 389 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 04:37:52.481003 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43b1c63-2c5b-4f87-955a-196e1ca39664 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:37:52.481525 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43b1c63-2c5b-4f87-955a-196e1ca39664 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:37:52.481615 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43b1c63-2c5b-4f87-955a-196e1ca39664 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:37:52.506036 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-b43b1c63-2c5b-4f87-955a-196e1ca39664 None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:37:52.507302 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-b43b1c63-2c5b-4f87-955a-196e1ca39664 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:37:52.637688 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1062/2146] 10.222.0.30 () {58 vars in 958 bytes} [Tue Sep 6 04:37:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 04:37:52.645969 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba562e69-ad26-4dc8-9376-38cd66d9b16d None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:37:52.646406 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba562e69-ad26-4dc8-9376-38cd66d9b16d None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:37:52.646584 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba562e69-ad26-4dc8-9376-38cd66d9b16d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:37:52.690459 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-ba562e69-ad26-4dc8-9376-38cd66d9b16d None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:37:52.691726 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-ba562e69-ad26-4dc8-9376-38cd66d9b16d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:37:52.706486 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1085/2147] 10.222.0.30 () {58 vars in 958 bytes} [Tue Sep 6 04:37:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 04:37:52.710749 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43b1c63-2c5b-4f87-955a-196e1ca39664 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:37:52.722343 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43b1c63-2c5b-4f87-955a-196e1ca39664 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:37:52.728703 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e461dc31-397c-4971-b737-a30b5b63a0a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:37:52.730002 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e461dc31-397c-4971-b737-a30b5b63a0a8 admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:37:52.730525 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e461dc31-397c-4971-b737-a30b5b63a0a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:37:52.730683 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e461dc31-397c-4971-b737-a30b5b63a0a8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:37:52.731304 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e461dc31-397c-4971-b737-a30b5b63a0a8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:37:52.731589 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e461dc31-397c-4971-b737-a30b5b63a0a8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:37:52.733211 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e461dc31-397c-4971-b737-a30b5b63a0a8 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:37:52.746421 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1063/2148] 10.222.0.30 () {58 vars in 1153 bytes} [Tue Sep 6 04:37:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:37:52.750726 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba562e69-ad26-4dc8-9376-38cd66d9b16d None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:37:52.761172 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba562e69-ad26-4dc8-9376-38cd66d9b16d None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:37:52.768867 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d2b46e-96f6-49ff-ab1d-a8c254150923 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:37:52.773477 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d2b46e-96f6-49ff-ab1d-a8c254150923 admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:37:52.773477 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d2b46e-96f6-49ff-ab1d-a8c254150923 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:37:52.773477 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d2b46e-96f6-49ff-ab1d-a8c254150923 admin admin] PATH_INFO: `/v3/projects` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:37:52.773477 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d2b46e-96f6-49ff-ab1d-a8c254150923 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:37:52.773477 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d2b46e-96f6-49ff-ab1d-a8c254150923 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:37:52.868076 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:37:52.897906 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1086/2149] 10.222.0.30 () {60 vars in 1153 bytes} [Tue Sep 6 04:37:52 2022] POST /identity/v3/projects => generated 387 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 04:37:52.905400 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e461dc31-397c-4971-b737-a30b5b63a0a8 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:37:52.916334 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e461dc31-397c-4971-b737-a30b5b63a0a8 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:37:52.981019 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b75c29-6b8a-49a8-b6a3-4c27f2c50cc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:37:52.981745 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b75c29-6b8a-49a8-b6a3-4c27f2c50cc5 admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:37:52.982022 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b75c29-6b8a-49a8-b6a3-4c27f2c50cc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:37:52.982291 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b75c29-6b8a-49a8-b6a3-4c27f2c50cc5 admin admin] PATH_INFO: `/v3/users` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:37:52.982941 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b75c29-6b8a-49a8-b6a3-4c27f2c50cc5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:37:52.984864 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b75c29-6b8a-49a8-b6a3-4c27f2c50cc5 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:37:53.076576 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1064/2150] 10.222.0.30 () {60 vars in 1144 bytes} [Tue Sep 6 04:37:52 2022] POST /identity/v3/users => generated 294 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 04:37:53.080962 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d2b46e-96f6-49ff-ab1d-a8c254150923 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:37:53.091438 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:37:53.092740 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d2b46e-96f6-49ff-ab1d-a8c254150923 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:37:53.099483 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d1fbbd-6d4c-487a-9ac9-fcbb3a55fd29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:37:53.101217 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d1fbbd-6d4c-487a-9ac9-fcbb3a55fd29 admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:37:53.101217 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d1fbbd-6d4c-487a-9ac9-fcbb3a55fd29 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:37:53.101217 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d1fbbd-6d4c-487a-9ac9-fcbb3a55fd29 admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:37:53.101658 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d1fbbd-6d4c-487a-9ac9-fcbb3a55fd29 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:37:53.104468 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d1fbbd-6d4c-487a-9ac9-fcbb3a55fd29 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:37:53.116101 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1087/2151] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Sep 6 04:37:53 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 04:37:53.120859 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b75c29-6b8a-49a8-b6a3-4c27f2c50cc5 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:37:53.132780 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b75c29-6b8a-49a8-b6a3-4c27f2c50cc5 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:37:53.138734 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc933138-9d06-4d81-a8a2-9724a86c6950 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:37:53.139377 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc933138-9d06-4d81-a8a2-9724a86c6950 admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:37:53.139658 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc933138-9d06-4d81-a8a2-9724a86c6950 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:37:53.139932 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc933138-9d06-4d81-a8a2-9724a86c6950 admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:37:53.140583 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc933138-9d06-4d81-a8a2-9724a86c6950 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:37:53.142243 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc933138-9d06-4d81-a8a2-9724a86c6950 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:37:53.159380 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1065/2152] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Sep 6 04:37:53 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 04:37:53.160871 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d1fbbd-6d4c-487a-9ac9-fcbb3a55fd29 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:37:53.173384 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d1fbbd-6d4c-487a-9ac9-fcbb3a55fd29 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:37:53.184034 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7817c79-d504-4487-8411-61c762ac9caa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:37:53.186898 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7817c79-d504-4487-8411-61c762ac9caa admin admin] REQUEST_METHOD: `PUT` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:37:53.186898 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7817c79-d504-4487-8411-61c762ac9caa admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:37:53.186898 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7817c79-d504-4487-8411-61c762ac9caa admin admin] PATH_INFO: `/v3/projects/24a0bf5b0509455d82a99ee7ee7e315e/users/8a6dbae675b54eb28469a263a89594a4/roles/29607b2de39f40d68bb6ceef64b8a732` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:37:53.203607 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7817c79-d504-4487-8411-61c762ac9caa admin admin] RBAC: Authorizing `identity:create_grant(project_id=24a0bf5b0509455d82a99ee7ee7e315e, user_id=8a6dbae675b54eb28469a263a89594a4, role_id=29607b2de39f40d68bb6ceef64b8a732)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:37:53.210588 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7817c79-d504-4487-8411-61c762ac9caa admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:37:53.321422 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1088/2153] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Sep 6 04:37:53 2022] PUT /identity/v3/projects/24a0bf5b0509455d82a99ee7ee7e315e/users/8a6dbae675b54eb28469a263a89594a4/roles/29607b2de39f40d68bb6ceef64b8a732 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 04:37:53.334060 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daac1a42-4479-48f8-9271-edbde3e34e08 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:37:53.334355 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daac1a42-4479-48f8-9271-edbde3e34e08 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:37:53.334630 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daac1a42-4479-48f8-9271-edbde3e34e08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:37:53.374333 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-daac1a42-4479-48f8-9271-edbde3e34e08 None None] MFA Rules not processed for user `8a6dbae675b54eb28469a263a89594a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:37:53.375816 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-daac1a42-4479-48f8-9271-edbde3e34e08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:37:53.468172 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1066/2154] 10.222.0.30 () {58 vars in 958 bytes} [Tue Sep 6 04:37:53 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 04:37:53.477042 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7817c79-d504-4487-8411-61c762ac9caa admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:37:53.572670 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7817c79-d504-4487-8411-61c762ac9caa admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:37:53.585348 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986453cb-a4b6-4a06-9338-50bd8580cea3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:37:53.587700 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986453cb-a4b6-4a06-9338-50bd8580cea3 admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:37:53.591146 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986453cb-a4b6-4a06-9338-50bd8580cea3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:37:53.591146 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986453cb-a4b6-4a06-9338-50bd8580cea3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:37:53.591827 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-986453cb-a4b6-4a06-9338-50bd8580cea3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:37:53.594905 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-986453cb-a4b6-4a06-9338-50bd8580cea3 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:37:53.648399 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:37:53.719679 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1089/2155] 10.222.0.30 () {60 vars in 1153 bytes} [Tue Sep 6 04:37:53 2022] POST /identity/v3/projects => generated 385 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 04:37:53.731217 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daac1a42-4479-48f8-9271-edbde3e34e08 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:37:53.750604 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daac1a42-4479-48f8-9271-edbde3e34e08 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:37:53.864305 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b18f97f-376b-48e2-8ea4-d86e4d8742b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:37:53.864966 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b18f97f-376b-48e2-8ea4-d86e4d8742b2 admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:37:53.865374 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b18f97f-376b-48e2-8ea4-d86e4d8742b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:37:53.865660 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b18f97f-376b-48e2-8ea4-d86e4d8742b2 admin admin] PATH_INFO: `/v3/users` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:37:53.866484 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b18f97f-376b-48e2-8ea4-d86e4d8742b2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:37:53.870395 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b18f97f-376b-48e2-8ea4-d86e4d8742b2 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:37:54.009068 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1067/2156] 10.222.0.30 () {60 vars in 1144 bytes} [Tue Sep 6 04:37:53 2022] POST /identity/v3/users => generated 299 bytes in 276 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 04:37:54.011248 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986453cb-a4b6-4a06-9338-50bd8580cea3 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:37:54.020869 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986453cb-a4b6-4a06-9338-50bd8580cea3 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:37:54.025865 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e81e60-e8eb-4d05-8957-430b2d1c1909 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:37:54.026649 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e81e60-e8eb-4d05-8957-430b2d1c1909 admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:37:54.027000 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e81e60-e8eb-4d05-8957-430b2d1c1909 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:37:54.027372 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e81e60-e8eb-4d05-8957-430b2d1c1909 admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:37:54.028040 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e81e60-e8eb-4d05-8957-430b2d1c1909 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:37:54.029721 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e81e60-e8eb-4d05-8957-430b2d1c1909 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:37:54.042942 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1090/2157] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Sep 6 04:37:54 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 04:37:54.048009 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b18f97f-376b-48e2-8ea4-d86e4d8742b2 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:37:54.059586 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b18f97f-376b-48e2-8ea4-d86e4d8742b2 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:37:54.065283 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3d439f-00d4-43f3-a192-0a8901fe46a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:37:54.066122 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3d439f-00d4-43f3-a192-0a8901fe46a3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:37:54.067378 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3d439f-00d4-43f3-a192-0a8901fe46a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:37:54.067378 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3d439f-00d4-43f3-a192-0a8901fe46a3 admin admin] PATH_INFO: `/v3/projects/642fb2f0a089443e98dc9bb99056e822/users/0befdacd30cc47368da4924c9c5ec7ad/roles/ea4c814713b149108e488de855b025e4` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:37:54.087787 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3d439f-00d4-43f3-a192-0a8901fe46a3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=642fb2f0a089443e98dc9bb99056e822, user_id=0befdacd30cc47368da4924c9c5ec7ad, role_id=ea4c814713b149108e488de855b025e4)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:37:54.097712 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3d439f-00d4-43f3-a192-0a8901fe46a3 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:37:54.196993 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1068/2158] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Sep 6 04:37:54 2022] PUT /identity/v3/projects/642fb2f0a089443e98dc9bb99056e822/users/0befdacd30cc47368da4924c9c5ec7ad/roles/ea4c814713b149108e488de855b025e4 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 04:37:54.201681 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e81e60-e8eb-4d05-8957-430b2d1c1909 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:37:54.212428 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e81e60-e8eb-4d05-8957-430b2d1c1909 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:37:54.283046 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39fe5dd-1807-4770-b424-3cc1260a57e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:37:54.283860 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39fe5dd-1807-4770-b424-3cc1260a57e0 admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:37:54.284258 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39fe5dd-1807-4770-b424-3cc1260a57e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:37:54.284635 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39fe5dd-1807-4770-b424-3cc1260a57e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:37:54.285457 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a39fe5dd-1807-4770-b424-3cc1260a57e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:37:54.287228 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a39fe5dd-1807-4770-b424-3cc1260a57e0 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:37:54.305895 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1091/2159] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Sep 6 04:37:54 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 06 04:37:54.310948 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3d439f-00d4-43f3-a192-0a8901fe46a3 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:37:54.322866 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3d439f-00d4-43f3-a192-0a8901fe46a3 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:37:54.328540 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca741306-9e3d-4040-bb84-78ba107458fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:37:54.331990 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca741306-9e3d-4040-bb84-78ba107458fb admin admin] REQUEST_METHOD: `PUT` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:37:54.332466 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca741306-9e3d-4040-bb84-78ba107458fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:37:54.332933 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca741306-9e3d-4040-bb84-78ba107458fb admin admin] PATH_INFO: `/v3/projects/642fb2f0a089443e98dc9bb99056e822/users/0befdacd30cc47368da4924c9c5ec7ad/roles/ea4c814713b149108e488de855b025e4` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:37:54.335284 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca741306-9e3d-4040-bb84-78ba107458fb admin admin] RBAC: Authorizing `identity:create_grant(project_id=642fb2f0a089443e98dc9bb99056e822, user_id=0befdacd30cc47368da4924c9c5ec7ad, role_id=ea4c814713b149108e488de855b025e4)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:37:54.338152 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca741306-9e3d-4040-bb84-78ba107458fb admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:37:54.349682 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1069/2160] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Sep 6 04:37:54 2022] PUT /identity/v3/projects/642fb2f0a089443e98dc9bb99056e822/users/0befdacd30cc47368da4924c9c5ec7ad/roles/ea4c814713b149108e488de855b025e4 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 04:37:54.363082 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe194bc-536b-4f61-81d7-ed6e2b78efbd None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:37:54.363391 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe194bc-536b-4f61-81d7-ed6e2b78efbd None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:37:54.363658 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe194bc-536b-4f61-81d7-ed6e2b78efbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:37:54.393976 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:37:54.422067 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-efe194bc-536b-4f61-81d7-ed6e2b78efbd None None] MFA Rules not processed for user `0befdacd30cc47368da4924c9c5ec7ad`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:37:54.422067 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-efe194bc-536b-4f61-81d7-ed6e2b78efbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:37:54.487653 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1092/2161] 10.222.0.30 () {58 vars in 958 bytes} [Tue Sep 6 04:37:54 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 04:37:54.500024 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca741306-9e3d-4040-bb84-78ba107458fb admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:37:54.501829 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-ca741306-9e3d-4040-bb84-78ba107458fb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:37:54.568509 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca741306-9e3d-4040-bb84-78ba107458fb admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:37:54.572218 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f637685-db82-4a8b-b2d2-d43e4f260454 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd09b5ab675634a2eb4b03755743f7d0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b707733ce65c42edbc1fd95ad0a1c06c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:37:54.572825 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f637685-db82-4a8b-b2d2-d43e4f260454 service cinder] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:37:54.573475 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f637685-db82-4a8b-b2d2-d43e4f260454 service cinder] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:37:54.573779 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f637685-db82-4a8b-b2d2-d43e4f260454 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:37:54.582745 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f637685-db82-4a8b-b2d2-d43e4f260454 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:37:54.584159 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f637685-db82-4a8b-b2d2-d43e4f260454 service cinder] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:37:54.604944 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1070/2162] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Sep 6 04:37:54 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 04:37:54.635310 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe194bc-536b-4f61-81d7-ed6e2b78efbd None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:37:54.715824 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe194bc-536b-4f61-81d7-ed6e2b78efbd None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:37:54.725494 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e10425-8f07-4a8a-a154-7a1c6b396bab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:37:54.731823 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e10425-8f07-4a8a-a154-7a1c6b396bab admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:37:54.731823 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e10425-8f07-4a8a-a154-7a1c6b396bab admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:37:54.731823 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e10425-8f07-4a8a-a154-7a1c6b396bab admin admin] PATH_INFO: `/v3/users/8a6dbae675b54eb28469a263a89594a4` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:37:54.731823 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e10425-8f07-4a8a-a154-7a1c6b396bab admin admin] RBAC: Authorizing `identity:delete_user(user_id=8a6dbae675b54eb28469a263a89594a4)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:37:54.731823 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e10425-8f07-4a8a-a154-7a1c6b396bab admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:37:54.971414 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-f0e10425-8f07-4a8a-a154-7a1c6b396bab admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8a6dbae675b54eb28469a263a89594a4'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:37:55.067795 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-f0e10425-8f07-4a8a-a154-7a1c6b396bab admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8a6dbae675b54eb28469a263a89594a4'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:37:55.068701 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-f0e10425-8f07-4a8a-a154-7a1c6b396bab admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8a6dbae675b54eb28469a263a89594a4'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:37:55.082624 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-f0e10425-8f07-4a8a-a154-7a1c6b396bab admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8a6dbae675b54eb28469a263a89594a4'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:37:55.090107 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-f0e10425-8f07-4a8a-a154-7a1c6b396bab admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8a6dbae675b54eb28469a263a89594a4'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:37:55.096549 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1093/2163] 10.222.0.30 () {58 vars in 1224 bytes} [Tue Sep 6 04:37:54 2022] DELETE /identity/v3/users/8a6dbae675b54eb28469a263a89594a4 => generated 0 bytes in 462 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 04:37:55.106711 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38a590c-2d64-44bf-8ccd-1d040f43a28c None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:37:55.106711 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38a590c-2d64-44bf-8ccd-1d040f43a28c None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:37:55.106711 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38a590c-2d64-44bf-8ccd-1d040f43a28c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:37:55.130178 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-a38a590c-2d64-44bf-8ccd-1d040f43a28c None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:37:55.131827 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-a38a590c-2d64-44bf-8ccd-1d040f43a28c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:37:55.168920 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:37:55.249088 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1071/2164] 10.222.0.30 () {58 vars in 958 bytes} [Tue Sep 6 04:37:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 04:37:55.255539 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e10425-8f07-4a8a-a154-7a1c6b396bab admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:37:55.257094 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-f0e10425-8f07-4a8a-a154-7a1c6b396bab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:37:55.321243 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:37:55.406590 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e10425-8f07-4a8a-a154-7a1c6b396bab admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:37:55.418171 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebdb993-c5c2-471e-9215-3113c37062af service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90492440f6a6471e8392361c098428aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b707733ce65c42edbc1fd95ad0a1c06c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:37:55.419261 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebdb993-c5c2-471e-9215-3113c37062af service neutron] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:37:55.420096 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebdb993-c5c2-471e-9215-3113c37062af service neutron] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:37:55.422699 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebdb993-c5c2-471e-9215-3113c37062af service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:37:55.459858 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cebdb993-c5c2-471e-9215-3113c37062af service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:37:55.459858 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cebdb993-c5c2-471e-9215-3113c37062af service neutron] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:37:55.483672 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38a590c-2d64-44bf-8ccd-1d040f43a28c None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:37:55.486118 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-a38a590c-2d64-44bf-8ccd-1d040f43a28c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:37:55.505505 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38a590c-2d64-44bf-8ccd-1d040f43a28c None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:37:55.511441 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1094/2165] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Sep 6 04:37:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 251 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 06 04:37:55.521134 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395d685a-5ff8-4125-9919-e2df996112c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:37:55.522074 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395d685a-5ff8-4125-9919-e2df996112c2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:37:55.522522 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395d685a-5ff8-4125-9919-e2df996112c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:37:55.522994 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395d685a-5ff8-4125-9919-e2df996112c2 admin admin] PATH_INFO: `/v3/users/b5ff6cccb0fd4573b5cfa22d808e3392` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:37:55.527939 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395d685a-5ff8-4125-9919-e2df996112c2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b5ff6cccb0fd4573b5cfa22d808e3392)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:37:55.530552 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395d685a-5ff8-4125-9919-e2df996112c2 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:37:55.831055 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-395d685a-5ff8-4125-9919-e2df996112c2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b5ff6cccb0fd4573b5cfa22d808e3392'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:37:55.948396 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-395d685a-5ff8-4125-9919-e2df996112c2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b5ff6cccb0fd4573b5cfa22d808e3392'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:37:55.949124 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-395d685a-5ff8-4125-9919-e2df996112c2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b5ff6cccb0fd4573b5cfa22d808e3392'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:37:55.961959 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-395d685a-5ff8-4125-9919-e2df996112c2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b5ff6cccb0fd4573b5cfa22d808e3392'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:37:55.981040 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-395d685a-5ff8-4125-9919-e2df996112c2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b5ff6cccb0fd4573b5cfa22d808e3392'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:37:55.989368 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1072/2166] 10.222.0.30 () {58 vars in 1224 bytes} [Tue Sep 6 04:37:55 2022] DELETE /identity/v3/users/b5ff6cccb0fd4573b5cfa22d808e3392 => generated 0 bytes in 508 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 04:37:55.994073 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08146bf1-f3bf-4cea-bfc3-7ba563ea56b8 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:37:55.994353 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08146bf1-f3bf-4cea-bfc3-7ba563ea56b8 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:37:55.994627 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08146bf1-f3bf-4cea-bfc3-7ba563ea56b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:37:56.015662 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-08146bf1-f3bf-4cea-bfc3-7ba563ea56b8 None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:37:56.017228 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-08146bf1-f3bf-4cea-bfc3-7ba563ea56b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:37:56.042842 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395d685a-5ff8-4125-9919-e2df996112c2 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:37:56.045088 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-395d685a-5ff8-4125-9919-e2df996112c2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:37:56.122029 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1095/2167] 10.222.0.30 () {58 vars in 958 bytes} [Tue Sep 6 04:37:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 04:37:56.135215 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08146bf1-f3bf-4cea-bfc3-7ba563ea56b8 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:37:56.137258 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-08146bf1-f3bf-4cea-bfc3-7ba563ea56b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:37:56.179038 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395d685a-5ff8-4125-9919-e2df996112c2 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:37:56.188847 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f73ba6b-937e-4668-94f9-016bbcb002da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:37:56.189536 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f73ba6b-937e-4668-94f9-016bbcb002da admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:37:56.192891 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f73ba6b-937e-4668-94f9-016bbcb002da admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:37:56.193176 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f73ba6b-937e-4668-94f9-016bbcb002da admin admin] PATH_INFO: `/v3/projects/24a0bf5b0509455d82a99ee7ee7e315e` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:37:56.196452 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f73ba6b-937e-4668-94f9-016bbcb002da admin admin] RBAC: Authorizing `identity:delete_project(project_id=24a0bf5b0509455d82a99ee7ee7e315e)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:37:56.199452 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f73ba6b-937e-4668-94f9-016bbcb002da admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:37:56.217634 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08146bf1-f3bf-4cea-bfc3-7ba563ea56b8 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:37:56.222065 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea67023-b542-42aa-a016-5e61676d686e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90492440f6a6471e8392361c098428aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b707733ce65c42edbc1fd95ad0a1c06c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:37:56.222967 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea67023-b542-42aa-a016-5e61676d686e service neutron] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:37:56.223448 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea67023-b542-42aa-a016-5e61676d686e service neutron] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:37:56.223932 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea67023-b542-42aa-a016-5e61676d686e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:37:56.236346 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea67023-b542-42aa-a016-5e61676d686e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:37:56.238118 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea67023-b542-42aa-a016-5e61676d686e service neutron] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:37:56.269541 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1096/2168] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Sep 6 04:37:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 06 04:37:56.346298 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:37:56.355502 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-8f73ba6b-937e-4668-94f9-016bbcb002da admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '24a0bf5b0509455d82a99ee7ee7e315e', 'request_id': 'req-8f73ba6b-937e-4668-94f9-016bbcb002da'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:37:56.435168 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-8f73ba6b-937e-4668-94f9-016bbcb002da admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '24a0bf5b0509455d82a99ee7ee7e315e', 'request_id': 'req-8f73ba6b-937e-4668-94f9-016bbcb002da'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:37:56.447625 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-8f73ba6b-937e-4668-94f9-016bbcb002da admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '24a0bf5b0509455d82a99ee7ee7e315e', 'request_id': 'req-8f73ba6b-937e-4668-94f9-016bbcb002da'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:37:56.464201 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-8f73ba6b-937e-4668-94f9-016bbcb002da admin admin] The token cache is being invalidate because project 24a0bf5b0509455d82a99ee7ee7e315e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81697) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 04:37:56.464499 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-8f73ba6b-937e-4668-94f9-016bbcb002da admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:37:56.465479 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-8f73ba6b-937e-4668-94f9-016bbcb002da admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:37:56.467826 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1073/2169] 10.222.0.30 () {58 vars in 1233 bytes} [Tue Sep 6 04:37:56 2022] DELETE /identity/v3/projects/24a0bf5b0509455d82a99ee7ee7e315e => generated 0 bytes in 426 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 04:37:56.482645 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea67023-b542-42aa-a016-5e61676d686e service neutron] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:37:56.494219 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea67023-b542-42aa-a016-5e61676d686e service neutron] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:37:56.519499 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f73ba6b-937e-4668-94f9-016bbcb002da admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:37:56.521974 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-8f73ba6b-937e-4668-94f9-016bbcb002da admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:37:56.584130 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:37:56.598868 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ea841f-14e3-4fce-b0ae-e9e357433fe4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:37:56.599682 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ea841f-14e3-4fce-b0ae-e9e357433fe4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:37:56.600154 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ea841f-14e3-4fce-b0ae-e9e357433fe4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:37:56.600435 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ea841f-14e3-4fce-b0ae-e9e357433fe4 admin admin] PATH_INFO: `/v3/users/0befdacd30cc47368da4924c9c5ec7ad` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:37:56.603311 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ea841f-14e3-4fce-b0ae-e9e357433fe4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0befdacd30cc47368da4924c9c5ec7ad)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:37:56.610897 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ea841f-14e3-4fce-b0ae-e9e357433fe4 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:37:56.635213 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f73ba6b-937e-4668-94f9-016bbcb002da admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:37:56.644689 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82da3609-709a-484d-966c-5f13ddb09d1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:37:56.649604 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82da3609-709a-484d-966c-5f13ddb09d1d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:37:56.649604 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82da3609-709a-484d-966c-5f13ddb09d1d admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:37:56.649604 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82da3609-709a-484d-966c-5f13ddb09d1d admin admin] PATH_INFO: `/v3/users/83b9a7b470f04e27a64d9cad114ed9a8` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:37:56.649604 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82da3609-709a-484d-966c-5f13ddb09d1d admin admin] RBAC: Authorizing `identity:delete_user(user_id=83b9a7b470f04e27a64d9cad114ed9a8)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:37:56.653028 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82da3609-709a-484d-966c-5f13ddb09d1d admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:37:56.856989 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-62ea841f-14e3-4fce-b0ae-e9e357433fe4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0befdacd30cc47368da4924c9c5ec7ad'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:37:56.936684 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-62ea841f-14e3-4fce-b0ae-e9e357433fe4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0befdacd30cc47368da4924c9c5ec7ad'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:37:56.937205 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-62ea841f-14e3-4fce-b0ae-e9e357433fe4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0befdacd30cc47368da4924c9c5ec7ad'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:37:56.943764 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-82da3609-709a-484d-966c-5f13ddb09d1d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '83b9a7b470f04e27a64d9cad114ed9a8'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:37:56.976770 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-62ea841f-14e3-4fce-b0ae-e9e357433fe4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0befdacd30cc47368da4924c9c5ec7ad'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:37:56.986436 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-62ea841f-14e3-4fce-b0ae-e9e357433fe4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0befdacd30cc47368da4924c9c5ec7ad'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:37:56.993563 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1097/2170] 10.222.0.30 () {58 vars in 1224 bytes} [Tue Sep 6 04:37:56 2022] DELETE /identity/v3/users/0befdacd30cc47368da4924c9c5ec7ad => generated 0 bytes in 511 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 04:37:56.997925 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ea841f-14e3-4fce-b0ae-e9e357433fe4 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:37:56.997925 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-62ea841f-14e3-4fce-b0ae-e9e357433fe4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:37:57.039381 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-82da3609-709a-484d-966c-5f13ddb09d1d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '83b9a7b470f04e27a64d9cad114ed9a8'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:37:57.040318 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-82da3609-709a-484d-966c-5f13ddb09d1d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '83b9a7b470f04e27a64d9cad114ed9a8'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:37:57.053230 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-82da3609-709a-484d-966c-5f13ddb09d1d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '83b9a7b470f04e27a64d9cad114ed9a8'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:37:57.077170 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-82da3609-709a-484d-966c-5f13ddb09d1d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '83b9a7b470f04e27a64d9cad114ed9a8'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:37:57.098800 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ea841f-14e3-4fce-b0ae-e9e357433fe4 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:37:57.110065 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1074/2171] 10.222.0.30 () {58 vars in 1224 bytes} [Tue Sep 6 04:37:56 2022] DELETE /identity/v3/users/83b9a7b470f04e27a64d9cad114ed9a8 => generated 0 bytes in 588 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 04:37:57.112626 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163c49e4-2fd5-4e29-852d-69ee084410de None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:37:57.112918 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163c49e4-2fd5-4e29-852d-69ee084410de None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:37:57.117644 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163c49e4-2fd5-4e29-852d-69ee084410de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:37:57.157822 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-163c49e4-2fd5-4e29-852d-69ee084410de None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:37:57.160069 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-163c49e4-2fd5-4e29-852d-69ee084410de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:37:57.216151 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c91fd8-9c4a-4cb2-a6f0-2107a128095f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:37:57.217278 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c91fd8-9c4a-4cb2-a6f0-2107a128095f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:37:57.218312 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c91fd8-9c4a-4cb2-a6f0-2107a128095f admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:37:57.218883 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c91fd8-9c4a-4cb2-a6f0-2107a128095f admin admin] PATH_INFO: `/v3/projects/5eded26134634335ab9ed9afc6067854` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:37:57.226348 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c91fd8-9c4a-4cb2-a6f0-2107a128095f admin admin] RBAC: Authorizing `identity:delete_project(project_id=5eded26134634335ab9ed9afc6067854)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:37:57.232234 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c91fd8-9c4a-4cb2-a6f0-2107a128095f admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:37:57.308656 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1075/2172] 10.222.0.30 () {58 vars in 958 bytes} [Tue Sep 6 04:37:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 04:37:57.325648 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:DELETE=14,keystone:SELECT=84,keystone:INSERT=2 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:37:57.336325 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163c49e4-2fd5-4e29-852d-69ee084410de None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:37:57.343316 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-163c49e4-2fd5-4e29-852d-69ee084410de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:37:57.373287 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:37:57.408513 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-c3c91fd8-9c4a-4cb2-a6f0-2107a128095f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5eded26134634335ab9ed9afc6067854', 'request_id': 'req-c3c91fd8-9c4a-4cb2-a6f0-2107a128095f'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:37:57.420410 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-c3c91fd8-9c4a-4cb2-a6f0-2107a128095f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5eded26134634335ab9ed9afc6067854', 'request_id': 'req-c3c91fd8-9c4a-4cb2-a6f0-2107a128095f'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:37:57.474020 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163c49e4-2fd5-4e29-852d-69ee084410de None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:37:57.501515 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-c3c91fd8-9c4a-4cb2-a6f0-2107a128095f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5eded26134634335ab9ed9afc6067854', 'request_id': 'req-c3c91fd8-9c4a-4cb2-a6f0-2107a128095f'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:37:57.516636 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-c3c91fd8-9c4a-4cb2-a6f0-2107a128095f admin admin] The token cache is being invalidate because project 5eded26134634335ab9ed9afc6067854 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81698) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 04:37:57.518947 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-c3c91fd8-9c4a-4cb2-a6f0-2107a128095f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:37:57.520274 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-c3c91fd8-9c4a-4cb2-a6f0-2107a128095f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:37:57.522051 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1098/2173] 10.222.0.30 () {58 vars in 1233 bytes} [Tue Sep 6 04:37:56 2022] DELETE /identity/v3/projects/5eded26134634335ab9ed9afc6067854 => generated 0 bytes in 528 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 04:37:57.529930 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b3cfac-3f4d-4477-813b-fbab3b3dccc8 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:37:57.530630 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b3cfac-3f4d-4477-813b-fbab3b3dccc8 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:37:57.530630 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b3cfac-3f4d-4477-813b-fbab3b3dccc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:37:57.543447 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa713af0-627a-4357-8188-4f36ad3a8525 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90492440f6a6471e8392361c098428aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b707733ce65c42edbc1fd95ad0a1c06c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:37:57.544173 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa713af0-627a-4357-8188-4f36ad3a8525 service neutron] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:37:57.544460 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa713af0-627a-4357-8188-4f36ad3a8525 service neutron] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:37:57.544744 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa713af0-627a-4357-8188-4f36ad3a8525 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:37:57.546703 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-aa713af0-627a-4357-8188-4f36ad3a8525 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:37:57.557188 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-88b3cfac-3f4d-4477-813b-fbab3b3dccc8 None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:37:57.559282 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-88b3cfac-3f4d-4477-813b-fbab3b3dccc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:37:57.651478 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1099/2174] 10.222.0.30 () {58 vars in 958 bytes} [Tue Sep 6 04:37:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 04:37:57.652161 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b3cfac-3f4d-4477-813b-fbab3b3dccc8 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:37:57.653793 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-88b3cfac-3f4d-4477-813b-fbab3b3dccc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:37:57.662850 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa713af0-627a-4357-8188-4f36ad3a8525 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:37:57.664345 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa713af0-627a-4357-8188-4f36ad3a8525 service neutron] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:37:57.669376 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b3cfac-3f4d-4477-813b-fbab3b3dccc8 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:37:57.685567 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a28902-084a-4771-a652-e606207fb4a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:37:57.685567 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a28902-084a-4771-a652-e606207fb4a7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:37:57.685567 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a28902-084a-4771-a652-e606207fb4a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:37:57.685567 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a28902-084a-4771-a652-e606207fb4a7 admin admin] PATH_INFO: `/v3/projects/642fb2f0a089443e98dc9bb99056e822` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:37:57.686279 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a28902-084a-4771-a652-e606207fb4a7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=642fb2f0a089443e98dc9bb99056e822)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:37:57.686279 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a28902-084a-4771-a652-e606207fb4a7 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:37:57.690678 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1076/2175] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Sep 6 04:37:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 04:37:57.692252 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d603de-6ff8-4374-83d6-5bfc625988bc None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:37:57.692549 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d603de-6ff8-4374-83d6-5bfc625988bc None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:37:57.692895 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d603de-6ff8-4374-83d6-5bfc625988bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:37:57.718692 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-70d603de-6ff8-4374-83d6-5bfc625988bc None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:37:57.720003 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-70d603de-6ff8-4374-83d6-5bfc625988bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:37:57.730764 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1077/2176] 10.222.0.30 () {58 vars in 958 bytes} [Tue Sep 6 04:37:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 04:37:57.735408 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d603de-6ff8-4374-83d6-5bfc625988bc None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:37:57.745556 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d603de-6ff8-4374-83d6-5bfc625988bc None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:37:57.750639 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96881601-145e-4f77-ad2b-858d51b8efdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:37:57.751267 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96881601-145e-4f77-ad2b-858d51b8efdf admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:37:57.753889 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96881601-145e-4f77-ad2b-858d51b8efdf admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:37:57.753889 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96881601-145e-4f77-ad2b-858d51b8efdf admin admin] PATH_INFO: `/v3/domains` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:37:57.753889 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96881601-145e-4f77-ad2b-858d51b8efdf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:37:57.753889 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96881601-145e-4f77-ad2b-858d51b8efdf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:37:57.754472 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96881601-145e-4f77-ad2b-858d51b8efdf admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:37:57.778755 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1078/2177] 10.222.0.30 () {58 vars in 1153 bytes} [Tue Sep 6 04:37:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 04:37:57.787183 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96881601-145e-4f77-ad2b-858d51b8efdf admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:37:57.793517 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96881601-145e-4f77-ad2b-858d51b8efdf admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:37:57.810332 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-c5a28902-084a-4771-a652-e606207fb4a7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '642fb2f0a089443e98dc9bb99056e822', 'request_id': 'req-c5a28902-084a-4771-a652-e606207fb4a7'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:37:57.816394 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-c5a28902-084a-4771-a652-e606207fb4a7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '642fb2f0a089443e98dc9bb99056e822', 'request_id': 'req-c5a28902-084a-4771-a652-e606207fb4a7'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:37:57.860515 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069e536f-a4d5-42ea-a768-ed9784212857 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:37:57.861153 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069e536f-a4d5-42ea-a768-ed9784212857 admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:37:57.861516 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069e536f-a4d5-42ea-a768-ed9784212857 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:37:57.861799 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069e536f-a4d5-42ea-a768-ed9784212857 admin admin] PATH_INFO: `/v3/projects` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:37:57.862445 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069e536f-a4d5-42ea-a768-ed9784212857 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:37:57.867476 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069e536f-a4d5-42ea-a768-ed9784212857 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:37:57.891601 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-c5a28902-084a-4771-a652-e606207fb4a7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '642fb2f0a089443e98dc9bb99056e822', 'request_id': 'req-c5a28902-084a-4771-a652-e606207fb4a7'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:37:57.925953 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-c5a28902-084a-4771-a652-e606207fb4a7 admin admin] The token cache is being invalidate because project 642fb2f0a089443e98dc9bb99056e822 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81698) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 04:37:57.926456 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-c5a28902-084a-4771-a652-e606207fb4a7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:37:57.927488 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-c5a28902-084a-4771-a652-e606207fb4a7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:37:57.929447 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1100/2178] 10.222.0.30 () {58 vars in 1233 bytes} [Tue Sep 6 04:37:57 2022] DELETE /identity/v3/projects/642fb2f0a089443e98dc9bb99056e822 => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 04:37:57.930256 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a28902-084a-4771-a652-e606207fb4a7 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:37:57.931763 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-c5a28902-084a-4771-a652-e606207fb4a7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:37:57.948839 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a28902-084a-4771-a652-e606207fb4a7 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:37:57.954347 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14747372-7cbb-4981-8584-4a08d6de6ddf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:37:57.954998 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14747372-7cbb-4981-8584-4a08d6de6ddf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:37:57.955279 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14747372-7cbb-4981-8584-4a08d6de6ddf admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:37:57.955553 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14747372-7cbb-4981-8584-4a08d6de6ddf admin admin] PATH_INFO: `/v3/users/42246e779460402595923ebc2a3cb38a` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:37:57.956821 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14747372-7cbb-4981-8584-4a08d6de6ddf admin admin] RBAC: Authorizing `identity:delete_user(user_id=42246e779460402595923ebc2a3cb38a)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:37:57.958864 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14747372-7cbb-4981-8584-4a08d6de6ddf admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:37:57.992616 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1079/2179] 10.222.0.30 () {60 vars in 1153 bytes} [Tue Sep 6 04:37:57 2022] POST /identity/v3/projects => generated 385 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 04:37:57.994175 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f6114c-8496-4dbf-8bf9-2c0956834369 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:37:57.994465 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f6114c-8496-4dbf-8bf9-2c0956834369 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:37:57.994740 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f6114c-8496-4dbf-8bf9-2c0956834369 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:37:58.017364 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-09f6114c-8496-4dbf-8bf9-2c0956834369 None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:37:58.018543 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-09f6114c-8496-4dbf-8bf9-2c0956834369 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:37:58.086387 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1080/2180] 10.222.0.30 () {58 vars in 958 bytes} [Tue Sep 6 04:37:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 04:37:58.087448 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f6114c-8496-4dbf-8bf9-2c0956834369 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:37:58.090262 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-09f6114c-8496-4dbf-8bf9-2c0956834369 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:37:58.102566 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f6114c-8496-4dbf-8bf9-2c0956834369 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:37:58.108139 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42352b4-16d7-49e1-8eff-8a6415488dc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:37:58.108762 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42352b4-16d7-49e1-8eff-8a6415488dc9 admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:37:58.109046 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42352b4-16d7-49e1-8eff-8a6415488dc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:37:58.109348 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42352b4-16d7-49e1-8eff-8a6415488dc9 admin admin] PATH_INFO: `/v3/users` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:37:58.110007 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42352b4-16d7-49e1-8eff-8a6415488dc9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:37:58.112070 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42352b4-16d7-49e1-8eff-8a6415488dc9 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:37:58.122871 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-14747372-7cbb-4981-8584-4a08d6de6ddf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '42246e779460402595923ebc2a3cb38a'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:37:58.165605 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:37:58.211129 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-14747372-7cbb-4981-8584-4a08d6de6ddf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '42246e779460402595923ebc2a3cb38a'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:37:58.214552 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-14747372-7cbb-4981-8584-4a08d6de6ddf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '42246e779460402595923ebc2a3cb38a'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:37:58.221345 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1081/2181] 10.222.0.30 () {60 vars in 1144 bytes} [Tue Sep 6 04:37:58 2022] POST /identity/v3/users => generated 293 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 04:37:58.221345 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6725e8-95da-47b5-b548-6eebb3550b18 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:37:58.221345 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6725e8-95da-47b5-b548-6eebb3550b18 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:37:58.221345 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6725e8-95da-47b5-b548-6eebb3550b18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:37:58.226659 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-14747372-7cbb-4981-8584-4a08d6de6ddf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '42246e779460402595923ebc2a3cb38a'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:37:58.235250 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-14747372-7cbb-4981-8584-4a08d6de6ddf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '42246e779460402595923ebc2a3cb38a'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:37:58.241955 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-ad6725e8-95da-47b5-b548-6eebb3550b18 None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:37:58.245579 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1101/2182] 10.222.0.30 () {58 vars in 1224 bytes} [Tue Sep 6 04:37:57 2022] DELETE /identity/v3/users/42246e779460402595923ebc2a3cb38a => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 04:37:58.245579 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14747372-7cbb-4981-8584-4a08d6de6ddf admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:37:58.245768 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-ad6725e8-95da-47b5-b548-6eebb3550b18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:37:58.245862 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-14747372-7cbb-4981-8584-4a08d6de6ddf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:37:58.305816 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1082/2183] 10.222.0.30 () {58 vars in 958 bytes} [Tue Sep 6 04:37:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 04:37:58.306814 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6725e8-95da-47b5-b548-6eebb3550b18 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:37:58.308225 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-ad6725e8-95da-47b5-b548-6eebb3550b18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:37:58.319790 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6725e8-95da-47b5-b548-6eebb3550b18 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:37:58.323839 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14747372-7cbb-4981-8584-4a08d6de6ddf admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:37:58.324836 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3b6249-07ef-4431-9ac2-05d0b0602346 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:37:58.325524 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ef2899-2d98-4339-a2e9-bc141d341a01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:37:58.325966 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ef2899-2d98-4339-a2e9-bc141d341a01 admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:37:58.326173 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3b6249-07ef-4431-9ac2-05d0b0602346 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:37:58.326289 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ef2899-2d98-4339-a2e9-bc141d341a01 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:37:58.326524 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ef2899-2d98-4339-a2e9-bc141d341a01 admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:37:58.326618 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3b6249-07ef-4431-9ac2-05d0b0602346 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:37:58.326803 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3b6249-07ef-4431-9ac2-05d0b0602346 admin admin] PATH_INFO: `/v3/projects/e37b1d9d7f084c31b1a7937665bfb48e` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:37:58.327181 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ef2899-2d98-4339-a2e9-bc141d341a01 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:37:58.328277 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3b6249-07ef-4431-9ac2-05d0b0602346 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e37b1d9d7f084c31b1a7937665bfb48e)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:37:58.328917 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ef2899-2d98-4339-a2e9-bc141d341a01 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:37:58.330367 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3b6249-07ef-4431-9ac2-05d0b0602346 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:37:58.341158 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1083/2184] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Sep 6 04:37:58 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 04:37:58.342460 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b840249-04cd-41cb-8e81-29db643ff3b5 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:37:58.342750 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b840249-04cd-41cb-8e81-29db643ff3b5 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:37:58.343026 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b840249-04cd-41cb-8e81-29db643ff3b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:37:58.365398 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-1b840249-04cd-41cb-8e81-29db643ff3b5 None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:37:58.366631 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-1b840249-04cd-41cb-8e81-29db643ff3b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:37:58.376261 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1084/2185] 10.222.0.30 () {58 vars in 958 bytes} [Tue Sep 6 04:37:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 04:37:58.377121 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b840249-04cd-41cb-8e81-29db643ff3b5 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:37:58.381452 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:DELETE=18,keystone:SELECT=80,keystone:INSERT=2 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:37:58.389797 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b840249-04cd-41cb-8e81-29db643ff3b5 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:37:58.394609 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f7e970-6a07-4deb-982a-ce3b4dd4ce75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:37:58.395226 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f7e970-6a07-4deb-982a-ce3b4dd4ce75 admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:37:58.395509 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f7e970-6a07-4deb-982a-ce3b4dd4ce75 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:37:58.395785 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f7e970-6a07-4deb-982a-ce3b4dd4ce75 admin admin] PATH_INFO: `/v3/domains` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:37:58.396439 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f7e970-6a07-4deb-982a-ce3b4dd4ce75 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:37:58.396743 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f7e970-6a07-4deb-982a-ce3b4dd4ce75 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:37:58.398535 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f7e970-6a07-4deb-982a-ce3b4dd4ce75 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:37:58.412204 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1085/2186] 10.222.0.30 () {58 vars in 1153 bytes} [Tue Sep 6 04:37:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:37:58.413057 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f7e970-6a07-4deb-982a-ce3b4dd4ce75 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:37:58.424221 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f7e970-6a07-4deb-982a-ce3b4dd4ce75 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:37:58.429354 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091694cd-569f-4837-ac05-e7784bb8a734 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:37:58.430318 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091694cd-569f-4837-ac05-e7784bb8a734 admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:37:58.430767 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091694cd-569f-4837-ac05-e7784bb8a734 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:37:58.431043 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091694cd-569f-4837-ac05-e7784bb8a734 admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:37:58.431694 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091694cd-569f-4837-ac05-e7784bb8a734 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:37:58.433893 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091694cd-569f-4837-ac05-e7784bb8a734 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:37:58.446282 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1086/2187] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Sep 6 04:37:58 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 04:37:58.446971 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091694cd-569f-4837-ac05-e7784bb8a734 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:37:58.448691 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-091694cd-569f-4837-ac05-e7784bb8a734 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:37:58.456734 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-fe3b6249-07ef-4431-9ac2-05d0b0602346 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e37b1d9d7f084c31b1a7937665bfb48e', 'request_id': 'req-fe3b6249-07ef-4431-9ac2-05d0b0602346'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:37:58.461645 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:37:58.464630 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-fe3b6249-07ef-4431-9ac2-05d0b0602346 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e37b1d9d7f084c31b1a7937665bfb48e', 'request_id': 'req-fe3b6249-07ef-4431-9ac2-05d0b0602346'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:37:58.513812 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-fe3b6249-07ef-4431-9ac2-05d0b0602346 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e37b1d9d7f084c31b1a7937665bfb48e', 'request_id': 'req-fe3b6249-07ef-4431-9ac2-05d0b0602346'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:37:58.519331 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091694cd-569f-4837-ac05-e7784bb8a734 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:37:58.533185 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-fe3b6249-07ef-4431-9ac2-05d0b0602346 admin admin] The token cache is being invalidate because project e37b1d9d7f084c31b1a7937665bfb48e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81698) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 04:37:58.533529 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-fe3b6249-07ef-4431-9ac2-05d0b0602346 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:37:58.538253 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-fe3b6249-07ef-4431-9ac2-05d0b0602346 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:37:58.538253 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1102/2188] 10.222.0.30 () {58 vars in 1233 bytes} [Tue Sep 6 04:37:58 2022] DELETE /identity/v3/projects/e37b1d9d7f084c31b1a7937665bfb48e => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 04:37:58.538253 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3b6249-07ef-4431-9ac2-05d0b0602346 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:37:58.538253 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-fe3b6249-07ef-4431-9ac2-05d0b0602346 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:37:58.583646 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc47434-fec6-4510-a780-160cb3a279cb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90492440f6a6471e8392361c098428aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b707733ce65c42edbc1fd95ad0a1c06c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:37:58.584451 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc47434-fec6-4510-a780-160cb3a279cb service neutron] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:37:58.584887 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc47434-fec6-4510-a780-160cb3a279cb service neutron] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:37:58.585167 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc47434-fec6-4510-a780-160cb3a279cb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:37:58.587272 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-6cc47434-fec6-4510-a780-160cb3a279cb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:37:58.622933 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3b6249-07ef-4431-9ac2-05d0b0602346 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:37:58.628776 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2316850c-7cbe-43af-80e6-22ee01a8d18f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:37:58.630120 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2316850c-7cbe-43af-80e6-22ee01a8d18f admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:37:58.630120 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2316850c-7cbe-43af-80e6-22ee01a8d18f admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:37:58.630120 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2316850c-7cbe-43af-80e6-22ee01a8d18f admin admin] PATH_INFO: `/v3/projects` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:37:58.630560 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2316850c-7cbe-43af-80e6-22ee01a8d18f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:37:58.632468 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2316850c-7cbe-43af-80e6-22ee01a8d18f admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:37:58.659329 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc47434-fec6-4510-a780-160cb3a279cb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:37:58.660871 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc47434-fec6-4510-a780-160cb3a279cb service neutron] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:37:58.672460 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1087/2189] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Sep 6 04:37:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 226 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 04:37:58.673293 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc47434-fec6-4510-a780-160cb3a279cb service neutron] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:37:58.674658 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-6cc47434-fec6-4510-a780-160cb3a279cb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:37:58.687423 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc47434-fec6-4510-a780-160cb3a279cb service neutron] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:37:58.692501 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e685971b-808a-46a9-aecc-e0ebb92d4eaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:37:58.693203 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e685971b-808a-46a9-aecc-e0ebb92d4eaf admin admin] REQUEST_METHOD: `PUT` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:37:58.693569 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e685971b-808a-46a9-aecc-e0ebb92d4eaf admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:37:58.693847 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e685971b-808a-46a9-aecc-e0ebb92d4eaf admin admin] PATH_INFO: `/v3/projects/2fdab554573b41e3a827b0bc99cd374b/users/93ad8fce80e846abaec07eeafc74b065/roles/29607b2de39f40d68bb6ceef64b8a732` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:37:58.711128 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1103/2190] 10.222.0.30 () {60 vars in 1153 bytes} [Tue Sep 6 04:37:58 2022] POST /identity/v3/projects => generated 391 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 04:37:58.713983 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9438606b-238b-482a-8177-229ed97efcc0 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:37:58.714279 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9438606b-238b-482a-8177-229ed97efcc0 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:37:58.714562 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9438606b-238b-482a-8177-229ed97efcc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:37:58.718024 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e685971b-808a-46a9-aecc-e0ebb92d4eaf admin admin] RBAC: Authorizing `identity:create_grant(project_id=2fdab554573b41e3a827b0bc99cd374b, user_id=93ad8fce80e846abaec07eeafc74b065, role_id=29607b2de39f40d68bb6ceef64b8a732)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:37:58.719239 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:37:58.720651 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e685971b-808a-46a9-aecc-e0ebb92d4eaf admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:37:58.738847 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-9438606b-238b-482a-8177-229ed97efcc0 None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:37:58.740143 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-9438606b-238b-482a-8177-229ed97efcc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:37:58.788807 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1088/2191] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Sep 6 04:37:58 2022] PUT /identity/v3/projects/2fdab554573b41e3a827b0bc99cd374b/users/93ad8fce80e846abaec07eeafc74b065/roles/29607b2de39f40d68bb6ceef64b8a732 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 04:37:58.797533 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e685971b-808a-46a9-aecc-e0ebb92d4eaf admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:37:58.800785 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1104/2192] 10.222.0.30 () {58 vars in 958 bytes} [Tue Sep 6 04:37:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 04:37:58.802085 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f8cb2c-b838-4187-81f5-73475be4128e None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:37:58.802402 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f8cb2c-b838-4187-81f5-73475be4128e None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:37:58.802693 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f8cb2c-b838-4187-81f5-73475be4128e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:37:58.834191 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-34f8cb2c-b838-4187-81f5-73475be4128e None None] MFA Rules not processed for user `93ad8fce80e846abaec07eeafc74b065`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:37:58.835345 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-34f8cb2c-b838-4187-81f5-73475be4128e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:37:58.855675 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:37:58.871941 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e685971b-808a-46a9-aecc-e0ebb92d4eaf admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:37:58.878294 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c2a5f1-d65b-4705-8bc2-d6808b05a9ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:37:58.881821 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c2a5f1-d65b-4705-8bc2-d6808b05a9ca admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:37:58.881821 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c2a5f1-d65b-4705-8bc2-d6808b05a9ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:37:58.881821 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c2a5f1-d65b-4705-8bc2-d6808b05a9ca admin admin] PATH_INFO: `/v3/users` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:37:58.881821 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c2a5f1-d65b-4705-8bc2-d6808b05a9ca admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:37:58.883223 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c2a5f1-d65b-4705-8bc2-d6808b05a9ca admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:37:58.914404 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1105/2193] 10.222.0.30 () {58 vars in 958 bytes} [Tue Sep 6 04:37:58 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 04:37:58.915473 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b7ec93-1fbe-4321-aea7-9e22202a9f21 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:37:58.915760 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b7ec93-1fbe-4321-aea7-9e22202a9f21 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:37:58.916037 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b7ec93-1fbe-4321-aea7-9e22202a9f21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:37:58.938562 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-52b7ec93-1fbe-4321-aea7-9e22202a9f21 None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:37:58.939813 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-52b7ec93-1fbe-4321-aea7-9e22202a9f21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:37:58.948177 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1106/2194] 10.222.0.30 () {58 vars in 958 bytes} [Tue Sep 6 04:37:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 04:37:58.949009 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b7ec93-1fbe-4321-aea7-9e22202a9f21 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:37:58.962884 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b7ec93-1fbe-4321-aea7-9e22202a9f21 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:37:58.965479 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a915630-d48c-43eb-8cce-f2e6179f8806 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:37:58.965971 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a915630-d48c-43eb-8cce-f2e6179f8806 admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:37:58.966256 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a915630-d48c-43eb-8cce-f2e6179f8806 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:37:58.966532 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a915630-d48c-43eb-8cce-f2e6179f8806 admin admin] PATH_INFO: `/v3/projects` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:37:58.967254 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a915630-d48c-43eb-8cce-f2e6179f8806 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:37:58.969594 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a915630-d48c-43eb-8cce-f2e6179f8806 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:37:59.022605 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1089/2195] 10.222.0.30 () {60 vars in 1144 bytes} [Tue Sep 6 04:37:58 2022] POST /identity/v3/users => generated 296 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 04:37:59.023554 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c2a5f1-d65b-4705-8bc2-d6808b05a9ca admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:37:59.034258 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c2a5f1-d65b-4705-8bc2-d6808b05a9ca admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:37:59.039304 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930d31cc-417e-4735-9846-7d0b6db3f0f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:37:59.039977 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930d31cc-417e-4735-9846-7d0b6db3f0f6 admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:37:59.040363 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930d31cc-417e-4735-9846-7d0b6db3f0f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:37:59.040723 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930d31cc-417e-4735-9846-7d0b6db3f0f6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:37:59.046102 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930d31cc-417e-4735-9846-7d0b6db3f0f6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:37:59.046102 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930d31cc-417e-4735-9846-7d0b6db3f0f6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:37:59.046102 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930d31cc-417e-4735-9846-7d0b6db3f0f6 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:37:59.056549 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1090/2196] 10.222.0.30 () {58 vars in 1153 bytes} [Tue Sep 6 04:37:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:37:59.058012 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930d31cc-417e-4735-9846-7d0b6db3f0f6 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:37:59.061362 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1107/2197] 10.222.0.30 () {60 vars in 1153 bytes} [Tue Sep 6 04:37:58 2022] POST /identity/v3/projects => generated 383 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 04:37:59.062523 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a915630-d48c-43eb-8cce-f2e6179f8806 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:37:59.068238 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930d31cc-417e-4735-9846-7d0b6db3f0f6 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:37:59.131242 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a915630-d48c-43eb-8cce-f2e6179f8806 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:37:59.135326 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38bf3c64-b140-4cb7-ae76-23143436d988 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:37:59.135326 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bf3c64-b140-4cb7-ae76-23143436d988 admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:37:59.135326 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bf3c64-b140-4cb7-ae76-23143436d988 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:37:59.135326 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bf3c64-b140-4cb7-ae76-23143436d988 admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:37:59.136261 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:37:59.136350 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38bf3c64-b140-4cb7-ae76-23143436d988 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:37:59.136529 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38bf3c64-b140-4cb7-ae76-23143436d988 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:37:59.138867 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92505933-1e85-40dd-846d-8ecff50b7b32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:37:59.139816 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92505933-1e85-40dd-846d-8ecff50b7b32 admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:37:59.140103 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92505933-1e85-40dd-846d-8ecff50b7b32 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:37:59.140408 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92505933-1e85-40dd-846d-8ecff50b7b32 admin admin] PATH_INFO: `/v3/projects` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:37:59.141147 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92505933-1e85-40dd-846d-8ecff50b7b32 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:37:59.143049 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92505933-1e85-40dd-846d-8ecff50b7b32 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:37:59.147267 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1091/2198] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Sep 6 04:37:59 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 04:37:59.148191 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38bf3c64-b140-4cb7-ae76-23143436d988 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:37:59.158902 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38bf3c64-b140-4cb7-ae76-23143436d988 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:37:59.163872 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d05f7f4-844d-47ab-bad2-56eb4e7ad4f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:37:59.164613 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d05f7f4-844d-47ab-bad2-56eb4e7ad4f9 admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:37:59.164941 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d05f7f4-844d-47ab-bad2-56eb4e7ad4f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:37:59.165251 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d05f7f4-844d-47ab-bad2-56eb4e7ad4f9 admin admin] PATH_INFO: `/v3/users` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:37:59.165951 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d05f7f4-844d-47ab-bad2-56eb4e7ad4f9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:37:59.168185 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d05f7f4-844d-47ab-bad2-56eb4e7ad4f9 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:37:59.237338 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1108/2199] 10.222.0.30 () {60 vars in 1153 bytes} [Tue Sep 6 04:37:59 2022] POST /identity/v3/projects => generated 375 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 04:37:59.237338 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92505933-1e85-40dd-846d-8ecff50b7b32 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:37:59.239134 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-92505933-1e85-40dd-846d-8ecff50b7b32 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:37:59.257569 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1092/2200] 10.222.0.30 () {60 vars in 1144 bytes} [Tue Sep 6 04:37:59 2022] POST /identity/v3/users => generated 292 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 04:37:59.258503 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d05f7f4-844d-47ab-bad2-56eb4e7ad4f9 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:37:59.278701 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d05f7f4-844d-47ab-bad2-56eb4e7ad4f9 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:37:59.319812 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92505933-1e85-40dd-846d-8ecff50b7b32 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:37:59.327788 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c49b7f-b621-45f6-b199-8d7228cbba9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:37:59.329246 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c49b7f-b621-45f6-b199-8d7228cbba9a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:37:59.333359 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c49b7f-b621-45f6-b199-8d7228cbba9a admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:37:59.333359 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c49b7f-b621-45f6-b199-8d7228cbba9a admin admin] PATH_INFO: `/v3/projects/45eb0ce3dada4e7db08a12ae882ca974` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:37:59.333359 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c49b7f-b621-45f6-b199-8d7228cbba9a admin admin] RBAC: Authorizing `identity:delete_project(project_id=45eb0ce3dada4e7db08a12ae882ca974)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:37:59.333698 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c49b7f-b621-45f6-b199-8d7228cbba9a admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:37:59.357160 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7551c997-e531-485e-a9ea-27f4366b0bc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:37:59.358330 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7551c997-e531-485e-a9ea-27f4366b0bc1 admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:37:59.358767 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7551c997-e531-485e-a9ea-27f4366b0bc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:37:59.361941 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7551c997-e531-485e-a9ea-27f4366b0bc1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:37:59.361941 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7551c997-e531-485e-a9ea-27f4366b0bc1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:37:59.361941 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7551c997-e531-485e-a9ea-27f4366b0bc1 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:37:59.373276 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1093/2201] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Sep 6 04:37:59 2022] GET /identity/v3/roles => generated 1420 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 04:37:59.374185 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7551c997-e531-485e-a9ea-27f4366b0bc1 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:37:59.384698 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7551c997-e531-485e-a9ea-27f4366b0bc1 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:37:59.390506 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9f7411-ab66-434b-a987-b2b53e16a4ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:37:59.391158 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9f7411-ab66-434b-a987-b2b53e16a4ea admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:37:59.391444 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9f7411-ab66-434b-a987-b2b53e16a4ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:37:59.391726 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9f7411-ab66-434b-a987-b2b53e16a4ea admin admin] PATH_INFO: `/v3/users` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:37:59.392368 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9f7411-ab66-434b-a987-b2b53e16a4ea admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:37:59.394573 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9f7411-ab66-434b-a987-b2b53e16a4ea admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:37:59.464913 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-30c49b7f-b621-45f6-b199-8d7228cbba9a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '45eb0ce3dada4e7db08a12ae882ca974', 'request_id': 'req-30c49b7f-b621-45f6-b199-8d7228cbba9a'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:37:59.471796 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-30c49b7f-b621-45f6-b199-8d7228cbba9a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '45eb0ce3dada4e7db08a12ae882ca974', 'request_id': 'req-30c49b7f-b621-45f6-b199-8d7228cbba9a'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:37:59.491760 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1094/2202] 10.222.0.30 () {60 vars in 1144 bytes} [Tue Sep 6 04:37:59 2022] POST /identity/v3/users => generated 288 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 04:37:59.492867 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9f7411-ab66-434b-a987-b2b53e16a4ea admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:37:59.506590 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9f7411-ab66-434b-a987-b2b53e16a4ea admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:37:59.530428 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=91,keystone:INSERT=9 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:37:59.538554 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-30c49b7f-b621-45f6-b199-8d7228cbba9a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '45eb0ce3dada4e7db08a12ae882ca974', 'request_id': 'req-30c49b7f-b621-45f6-b199-8d7228cbba9a'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:37:59.553684 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-30c49b7f-b621-45f6-b199-8d7228cbba9a admin admin] The token cache is being invalidate because project 45eb0ce3dada4e7db08a12ae882ca974 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81698) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 04:37:59.554097 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-30c49b7f-b621-45f6-b199-8d7228cbba9a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:37:59.554722 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-30c49b7f-b621-45f6-b199-8d7228cbba9a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:37:59.556492 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1109/2203] 10.222.0.30 () {58 vars in 1233 bytes} [Tue Sep 6 04:37:59 2022] DELETE /identity/v3/projects/45eb0ce3dada4e7db08a12ae882ca974 => generated 0 bytes in 320 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 04:37:59.557920 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c49b7f-b621-45f6-b199-8d7228cbba9a admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:37:59.567232 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-30c49b7f-b621-45f6-b199-8d7228cbba9a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:37:59.584179 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2260db4e-453a-4402-9c5f-e78aa7670827 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:37:59.584992 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2260db4e-453a-4402-9c5f-e78aa7670827 admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:37:59.585398 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2260db4e-453a-4402-9c5f-e78aa7670827 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:37:59.585898 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2260db4e-453a-4402-9c5f-e78aa7670827 admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:37:59.586658 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2260db4e-453a-4402-9c5f-e78aa7670827 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:37:59.588402 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2260db4e-453a-4402-9c5f-e78aa7670827 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:37:59.605682 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1095/2204] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Sep 6 04:37:59 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 04:37:59.606836 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2260db4e-453a-4402-9c5f-e78aa7670827 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:37:59.609490 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-2260db4e-453a-4402-9c5f-e78aa7670827 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:37:59.647820 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2260db4e-453a-4402-9c5f-e78aa7670827 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:37:59.655437 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80ba669-bfc1-48b9-a2c7-4cff3b80331e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:37:59.656080 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80ba669-bfc1-48b9-a2c7-4cff3b80331e admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:37:59.656363 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80ba669-bfc1-48b9-a2c7-4cff3b80331e admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:37:59.656637 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80ba669-bfc1-48b9-a2c7-4cff3b80331e admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:37:59.661754 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80ba669-bfc1-48b9-a2c7-4cff3b80331e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:37:59.661754 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80ba669-bfc1-48b9-a2c7-4cff3b80331e admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:37:59.677206 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1096/2205] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Sep 6 04:37:59 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 04:37:59.677206 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80ba669-bfc1-48b9-a2c7-4cff3b80331e admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:37:59.677654 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-c80ba669-bfc1-48b9-a2c7-4cff3b80331e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:37:59.694420 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80ba669-bfc1-48b9-a2c7-4cff3b80331e admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:37:59.699886 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2bbad6-afe4-47e9-828e-890cab88afe0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:37:59.700652 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2bbad6-afe4-47e9-828e-890cab88afe0 admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:37:59.700931 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2bbad6-afe4-47e9-828e-890cab88afe0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:37:59.701207 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2bbad6-afe4-47e9-828e-890cab88afe0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:37:59.701915 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2bbad6-afe4-47e9-828e-890cab88afe0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:37:59.703954 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2bbad6-afe4-47e9-828e-890cab88afe0 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:37:59.717092 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:37:59.732390 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1097/2206] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Sep 6 04:37:59 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 04:37:59.733295 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2bbad6-afe4-47e9-828e-890cab88afe0 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:37:59.758308 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2bbad6-afe4-47e9-828e-890cab88afe0 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:37:59.764412 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6fa554f-d61d-4a26-b748-347fc7389516 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:37:59.765055 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fa554f-d61d-4a26-b748-347fc7389516 admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:37:59.765364 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fa554f-d61d-4a26-b748-347fc7389516 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:37:59.765582 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c49b7f-b621-45f6-b199-8d7228cbba9a admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:37:59.766803 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fa554f-d61d-4a26-b748-347fc7389516 admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:37:59.766803 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6fa554f-d61d-4a26-b748-347fc7389516 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:37:59.767905 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6fa554f-d61d-4a26-b748-347fc7389516 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:37:59.779457 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d55da7b-83c2-4e6e-8d88-b1f1e46de28b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:37:59.780140 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d55da7b-83c2-4e6e-8d88-b1f1e46de28b admin admin] REQUEST_METHOD: `PUT` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:37:59.780433 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d55da7b-83c2-4e6e-8d88-b1f1e46de28b admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:37:59.787742 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d55da7b-83c2-4e6e-8d88-b1f1e46de28b admin admin] PATH_INFO: `/v3/projects/1b042cdece0a4ea3b9b9aa9c064218a8/users/fc58e2aa72f342f5ad66f60c2bb80926/roles/29607b2de39f40d68bb6ceef64b8a732` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:37:59.791369 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1098/2207] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Sep 6 04:37:59 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 04:37:59.793065 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6fa554f-d61d-4a26-b748-347fc7389516 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:37:59.818124 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6fa554f-d61d-4a26-b748-347fc7389516 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:37:59.819636 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d55da7b-83c2-4e6e-8d88-b1f1e46de28b admin admin] RBAC: Authorizing `identity:create_grant(project_id=1b042cdece0a4ea3b9b9aa9c064218a8, user_id=fc58e2aa72f342f5ad66f60c2bb80926, role_id=29607b2de39f40d68bb6ceef64b8a732)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:37:59.822375 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d55da7b-83c2-4e6e-8d88-b1f1e46de28b admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:37:59.830279 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03ee2d1-6915-49f7-b4ff-ea36bf3a87d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:37:59.830987 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03ee2d1-6915-49f7-b4ff-ea36bf3a87d7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:37:59.831263 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03ee2d1-6915-49f7-b4ff-ea36bf3a87d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:37:59.831534 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03ee2d1-6915-49f7-b4ff-ea36bf3a87d7 admin admin] PATH_INFO: `/v3/projects/5f2bb0dd0dac49c6b93117c169be1ce1/users/23e117b9d61340c9a59420cfa69d4c01/roles/29607b2de39f40d68bb6ceef64b8a732` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:37:59.850558 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03ee2d1-6915-49f7-b4ff-ea36bf3a87d7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5f2bb0dd0dac49c6b93117c169be1ce1, user_id=23e117b9d61340c9a59420cfa69d4c01, role_id=29607b2de39f40d68bb6ceef64b8a732)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:37:59.853433 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03ee2d1-6915-49f7-b4ff-ea36bf3a87d7 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:37:59.891409 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1110/2208] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Sep 6 04:37:59 2022] PUT /identity/v3/projects/1b042cdece0a4ea3b9b9aa9c064218a8/users/fc58e2aa72f342f5ad66f60c2bb80926/roles/29607b2de39f40d68bb6ceef64b8a732 => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 04:37:59.892224 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d55da7b-83c2-4e6e-8d88-b1f1e46de28b admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:37:59.944459 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1099/2209] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Sep 6 04:37:59 2022] PUT /identity/v3/projects/5f2bb0dd0dac49c6b93117c169be1ce1/users/23e117b9d61340c9a59420cfa69d4c01/roles/29607b2de39f40d68bb6ceef64b8a732 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 04:37:59.945888 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79c4ec4-d707-4a3a-a8e4-0b20d069b8b3 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:37:59.946237 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79c4ec4-d707-4a3a-a8e4-0b20d069b8b3 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:37:59.946709 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79c4ec4-d707-4a3a-a8e4-0b20d069b8b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:37:59.957266 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d55da7b-83c2-4e6e-8d88-b1f1e46de28b admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:37:59.985591 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-a79c4ec4-d707-4a3a-a8e4-0b20d069b8b3 None None] MFA Rules not processed for user `fc58e2aa72f342f5ad66f60c2bb80926`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:37:59.986851 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-a79c4ec4-d707-4a3a-a8e4-0b20d069b8b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:38:00.036158 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011d7705-04ee-4cbd-9b39-b6f2c87c959e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:38:00.036837 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011d7705-04ee-4cbd-9b39-b6f2c87c959e admin admin] REQUEST_METHOD: `PUT` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:38:00.037119 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011d7705-04ee-4cbd-9b39-b6f2c87c959e admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:38:00.037520 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011d7705-04ee-4cbd-9b39-b6f2c87c959e admin admin] PATH_INFO: `/v3/projects/213ccef9365044dc854b6da80f9c9b37/users/fc6eb1c7a71b41288f1ec882c45f261c/roles/29607b2de39f40d68bb6ceef64b8a732` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:38:00.054260 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011d7705-04ee-4cbd-9b39-b6f2c87c959e admin admin] RBAC: Authorizing `identity:create_grant(project_id=213ccef9365044dc854b6da80f9c9b37, user_id=fc6eb1c7a71b41288f1ec882c45f261c, role_id=29607b2de39f40d68bb6ceef64b8a732)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:38:00.056974 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011d7705-04ee-4cbd-9b39-b6f2c87c959e admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:38:00.059865 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1100/2210] 10.222.0.30 () {58 vars in 958 bytes} [Tue Sep 6 04:37:59 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 04:38:00.061241 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188e1896-6aff-471c-8420-baf3fc9e6f96 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:38:00.061567 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188e1896-6aff-471c-8420-baf3fc9e6f96 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:38:00.061845 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188e1896-6aff-471c-8420-baf3fc9e6f96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:38:00.090643 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-188e1896-6aff-471c-8420-baf3fc9e6f96 None None] MFA Rules not processed for user `23e117b9d61340c9a59420cfa69d4c01`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:38:00.091960 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-188e1896-6aff-471c-8420-baf3fc9e6f96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:38:00.109273 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1111/2211] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Sep 6 04:37:59 2022] PUT /identity/v3/projects/213ccef9365044dc854b6da80f9c9b37/users/fc6eb1c7a71b41288f1ec882c45f261c/roles/29607b2de39f40d68bb6ceef64b8a732 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 04:38:00.114887 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011d7705-04ee-4cbd-9b39-b6f2c87c959e admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:38:00.147795 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:38:00.174413 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1101/2212] 10.222.0.30 () {58 vars in 958 bytes} [Tue Sep 6 04:38:00 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 04:38:00.175998 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e938ab2-f0ed-498e-a2bd-f98554cf47be None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:38:00.176342 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e938ab2-f0ed-498e-a2bd-f98554cf47be None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:38:00.176679 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e938ab2-f0ed-498e-a2bd-f98554cf47be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:38:00.180023 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:38:00.184587 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011d7705-04ee-4cbd-9b39-b6f2c87c959e admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:38:00.190484 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b32160b-edf6-41ec-964f-91f54eb7b3bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:38:00.191100 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b32160b-edf6-41ec-964f-91f54eb7b3bf admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:38:00.191381 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b32160b-edf6-41ec-964f-91f54eb7b3bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:38:00.197579 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b32160b-edf6-41ec-964f-91f54eb7b3bf admin admin] PATH_INFO: `/v3/projects` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:38:00.197579 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b32160b-edf6-41ec-964f-91f54eb7b3bf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:38:00.197579 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b32160b-edf6-41ec-964f-91f54eb7b3bf admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:38:00.217245 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-2e938ab2-f0ed-498e-a2bd-f98554cf47be None None] MFA Rules not processed for user `fc6eb1c7a71b41288f1ec882c45f261c`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:38:00.218537 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-2e938ab2-f0ed-498e-a2bd-f98554cf47be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:38:00.288053 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1102/2213] 10.222.0.30 () {58 vars in 958 bytes} [Tue Sep 6 04:38:00 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 04:38:00.290419 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e938ab2-f0ed-498e-a2bd-f98554cf47be None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:38:00.293700 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1112/2214] 10.222.0.30 () {60 vars in 1153 bytes} [Tue Sep 6 04:38:00 2022] POST /identity/v3/projects => generated 391 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 04:38:00.297758 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b32160b-edf6-41ec-964f-91f54eb7b3bf admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:38:00.304295 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e938ab2-f0ed-498e-a2bd-f98554cf47be None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:38:00.364620 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b32160b-edf6-41ec-964f-91f54eb7b3bf admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:38:00.372915 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c02b4d5-1b1d-4ada-afca-10e59967047d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:38:00.374219 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c02b4d5-1b1d-4ada-afca-10e59967047d admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:38:00.375125 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c02b4d5-1b1d-4ada-afca-10e59967047d admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:38:00.375403 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c02b4d5-1b1d-4ada-afca-10e59967047d admin admin] PATH_INFO: `/v3/users` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:38:00.380639 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c02b4d5-1b1d-4ada-afca-10e59967047d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:38:00.380639 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c02b4d5-1b1d-4ada-afca-10e59967047d admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:38:00.380812 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d131498d-bfcb-455e-ada7-9748a06f1074 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:38:00.380812 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d131498d-bfcb-455e-ada7-9748a06f1074 admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:38:00.380812 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d131498d-bfcb-455e-ada7-9748a06f1074 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:38:00.380812 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d131498d-bfcb-455e-ada7-9748a06f1074 admin admin] PATH_INFO: `/v3/projects` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:38:00.381578 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d131498d-bfcb-455e-ada7-9748a06f1074 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:38:00.383162 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d131498d-bfcb-455e-ada7-9748a06f1074 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:38:00.492312 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1113/2215] 10.222.0.30 () {60 vars in 1144 bytes} [Tue Sep 6 04:38:00 2022] POST /identity/v3/users => generated 302 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 04:38:00.493108 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1103/2216] 10.222.0.30 () {60 vars in 1153 bytes} [Tue Sep 6 04:38:00 2022] POST /identity/v3/projects => generated 385 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 04:38:00.494080 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d131498d-bfcb-455e-ada7-9748a06f1074 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:38:00.505598 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-d131498d-bfcb-455e-ada7-9748a06f1074 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:38:00.516049 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c02b4d5-1b1d-4ada-afca-10e59967047d admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:38:00.529540 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c02b4d5-1b1d-4ada-afca-10e59967047d admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:38:00.562353 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d131498d-bfcb-455e-ada7-9748a06f1074 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:38:00.567413 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247d5837-ac36-4225-a909-bc643e0ae45c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd09b5ab675634a2eb4b03755743f7d0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b707733ce65c42edbc1fd95ad0a1c06c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:38:00.568041 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247d5837-ac36-4225-a909-bc643e0ae45c service cinder] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:38:00.568370 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247d5837-ac36-4225-a909-bc643e0ae45c service cinder] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:38:00.568656 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247d5837-ac36-4225-a909-bc643e0ae45c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:38:00.578120 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:38:00.594615 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be87684-6030-4df3-9a9a-cec041eac3d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:38:00.595246 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be87684-6030-4df3-9a9a-cec041eac3d3 admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:38:00.595527 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be87684-6030-4df3-9a9a-cec041eac3d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:38:00.595801 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be87684-6030-4df3-9a9a-cec041eac3d3 admin admin] PATH_INFO: `/v3/users` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:38:00.596452 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be87684-6030-4df3-9a9a-cec041eac3d3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:38:00.598491 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be87684-6030-4df3-9a9a-cec041eac3d3 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:38:00.636709 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-247d5837-ac36-4225-a909-bc643e0ae45c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:38:00.638396 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-247d5837-ac36-4225-a909-bc643e0ae45c service cinder] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:38:00.647469 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1104/2217] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Sep 6 04:38:00 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 04:38:00.648739 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247d5837-ac36-4225-a909-bc643e0ae45c service cinder] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:38:00.659340 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247d5837-ac36-4225-a909-bc643e0ae45c service cinder] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:38:00.664183 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71fac364-fdf3-4ce0-9dd9-578e8f559dd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:38:00.664842 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fac364-fdf3-4ce0-9dd9-578e8f559dd6 admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:38:00.665124 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fac364-fdf3-4ce0-9dd9-578e8f559dd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:38:00.665427 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fac364-fdf3-4ce0-9dd9-578e8f559dd6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:38:00.666092 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71fac364-fdf3-4ce0-9dd9-578e8f559dd6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:38:00.667668 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71fac364-fdf3-4ce0-9dd9-578e8f559dd6 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:38:00.680139 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1105/2218] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Sep 6 04:38:00 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 04:38:00.683014 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71fac364-fdf3-4ce0-9dd9-578e8f559dd6 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:38:00.688431 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1114/2219] 10.222.0.30 () {60 vars in 1144 bytes} [Tue Sep 6 04:38:00 2022] POST /identity/v3/users => generated 299 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 04:38:00.693785 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be87684-6030-4df3-9a9a-cec041eac3d3 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:38:00.695402 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71fac364-fdf3-4ce0-9dd9-578e8f559dd6 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:38:00.701807 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c05912c-6d6b-47f9-aefb-1436bc46f260 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:38:00.702511 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c05912c-6d6b-47f9-aefb-1436bc46f260 admin admin] REQUEST_METHOD: `PUT` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:38:00.702794 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c05912c-6d6b-47f9-aefb-1436bc46f260 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:38:00.703071 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c05912c-6d6b-47f9-aefb-1436bc46f260 admin admin] PATH_INFO: `/v3/projects/5d89bb133bec4b0ca1f716c742142507/users/5249a2ddfc1a488994bc8c220f14db4d/roles/ea4c814713b149108e488de855b025e4` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:38:00.704399 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be87684-6030-4df3-9a9a-cec041eac3d3 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:38:00.721345 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25137ab5-fd20-4f92-a2eb-756e446e1436 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:38:00.721345 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25137ab5-fd20-4f92-a2eb-756e446e1436 admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:38:00.721345 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25137ab5-fd20-4f92-a2eb-756e446e1436 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:38:00.721345 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25137ab5-fd20-4f92-a2eb-756e446e1436 admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:38:00.721978 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25137ab5-fd20-4f92-a2eb-756e446e1436 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:38:00.721978 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25137ab5-fd20-4f92-a2eb-756e446e1436 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:38:00.724840 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c05912c-6d6b-47f9-aefb-1436bc46f260 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5d89bb133bec4b0ca1f716c742142507, user_id=5249a2ddfc1a488994bc8c220f14db4d, role_id=ea4c814713b149108e488de855b025e4)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:38:00.727669 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c05912c-6d6b-47f9-aefb-1436bc46f260 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:38:00.733359 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1115/2220] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Sep 6 04:38:00 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 06 04:38:00.736977 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25137ab5-fd20-4f92-a2eb-756e446e1436 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:38:00.747963 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25137ab5-fd20-4f92-a2eb-756e446e1436 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:38:00.752973 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8747566b-d229-48d6-a1fb-443704368422 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:38:00.753653 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8747566b-d229-48d6-a1fb-443704368422 admin admin] REQUEST_METHOD: `PUT` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:38:00.753931 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8747566b-d229-48d6-a1fb-443704368422 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:38:00.754225 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8747566b-d229-48d6-a1fb-443704368422 admin admin] PATH_INFO: `/v3/projects/8332c70b0109400795a3382bcf1b6705/users/2f1102c965074ac2b69809af842e3e94/roles/ea4c814713b149108e488de855b025e4` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:38:00.773122 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8747566b-d229-48d6-a1fb-443704368422 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8332c70b0109400795a3382bcf1b6705, user_id=2f1102c965074ac2b69809af842e3e94, role_id=ea4c814713b149108e488de855b025e4)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:38:00.776880 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8747566b-d229-48d6-a1fb-443704368422 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:38:00.796537 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1106/2221] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Sep 6 04:38:00 2022] PUT /identity/v3/projects/5d89bb133bec4b0ca1f716c742142507/users/5249a2ddfc1a488994bc8c220f14db4d/roles/ea4c814713b149108e488de855b025e4 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 04:38:00.800533 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c05912c-6d6b-47f9-aefb-1436bc46f260 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:38:00.841942 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1116/2222] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Sep 6 04:38:00 2022] PUT /identity/v3/projects/8332c70b0109400795a3382bcf1b6705/users/2f1102c965074ac2b69809af842e3e94/roles/ea4c814713b149108e488de855b025e4 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 04:38:00.848429 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8747566b-d229-48d6-a1fb-443704368422 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:38:00.876551 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c05912c-6d6b-47f9-aefb-1436bc46f260 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:38:00.880878 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:38:00.943678 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8747566b-d229-48d6-a1fb-443704368422 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:38:00.949759 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66c9c9f-6014-4b93-b22a-40a9be489c8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:38:00.950538 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66c9c9f-6014-4b93-b22a-40a9be489c8d admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:38:00.950922 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66c9c9f-6014-4b93-b22a-40a9be489c8d admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:38:00.951303 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66c9c9f-6014-4b93-b22a-40a9be489c8d admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:38:00.952058 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66c9c9f-6014-4b93-b22a-40a9be489c8d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:38:00.952175 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c7ba3b-a980-4c95-823c-d54a326323c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:38:00.952716 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c7ba3b-a980-4c95-823c-d54a326323c4 admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:38:00.953796 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c7ba3b-a980-4c95-823c-d54a326323c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:38:00.953796 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c7ba3b-a980-4c95-823c-d54a326323c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:38:00.953986 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c7ba3b-a980-4c95-823c-d54a326323c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:38:00.954249 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66c9c9f-6014-4b93-b22a-40a9be489c8d admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:38:00.955576 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c7ba3b-a980-4c95-823c-d54a326323c4 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:38:00.971472 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1117/2223] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Sep 6 04:38:00 2022] GET /identity/v3/roles => generated 1420 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 06 04:38:00.973483 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1107/2224] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Sep 6 04:38:00 2022] GET /identity/v3/roles => generated 1420 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 04:38:00.973837 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66c9c9f-6014-4b93-b22a-40a9be489c8d admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:38:00.975624 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c7ba3b-a980-4c95-823c-d54a326323c4 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:38:00.985441 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66c9c9f-6014-4b93-b22a-40a9be489c8d admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:38:00.988342 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c7ba3b-a980-4c95-823c-d54a326323c4 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:38:00.994136 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92c626f-4dc9-40fe-975d-26239bf192fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:38:00.995568 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92c626f-4dc9-40fe-975d-26239bf192fb admin admin] REQUEST_METHOD: `PUT` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:38:00.995774 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a31d26b-719f-4ee9-ad70-756c5d2f5732 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:38:00.995993 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92c626f-4dc9-40fe-975d-26239bf192fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:38:00.996621 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92c626f-4dc9-40fe-975d-26239bf192fb admin admin] PATH_INFO: `/v3/projects/8332c70b0109400795a3382bcf1b6705/users/2f1102c965074ac2b69809af842e3e94/roles/ea4c814713b149108e488de855b025e4` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:38:00.996905 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a31d26b-719f-4ee9-ad70-756c5d2f5732 admin admin] REQUEST_METHOD: `PUT` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:38:00.997129 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a31d26b-719f-4ee9-ad70-756c5d2f5732 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:38:00.997532 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a31d26b-719f-4ee9-ad70-756c5d2f5732 admin admin] PATH_INFO: `/v3/projects/5d89bb133bec4b0ca1f716c742142507/users/5249a2ddfc1a488994bc8c220f14db4d/roles/ea4c814713b149108e488de855b025e4` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:38:00.998918 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92c626f-4dc9-40fe-975d-26239bf192fb admin admin] RBAC: Authorizing `identity:create_grant(project_id=8332c70b0109400795a3382bcf1b6705, user_id=2f1102c965074ac2b69809af842e3e94, role_id=ea4c814713b149108e488de855b025e4)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:38:01.000594 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a31d26b-719f-4ee9-ad70-756c5d2f5732 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5d89bb133bec4b0ca1f716c742142507, user_id=5249a2ddfc1a488994bc8c220f14db4d, role_id=ea4c814713b149108e488de855b025e4)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:38:01.001864 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92c626f-4dc9-40fe-975d-26239bf192fb admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:38:01.006207 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a31d26b-719f-4ee9-ad70-756c5d2f5732 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:38:01.014360 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1118/2225] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Sep 6 04:38:00 2022] PUT /identity/v3/projects/8332c70b0109400795a3382bcf1b6705/users/2f1102c965074ac2b69809af842e3e94/roles/ea4c814713b149108e488de855b025e4 => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 04:38:01.021111 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1108/2226] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Sep 6 04:38:00 2022] PUT /identity/v3/projects/5d89bb133bec4b0ca1f716c742142507/users/5249a2ddfc1a488994bc8c220f14db4d/roles/ea4c814713b149108e488de855b025e4 => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 04:38:01.032387 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957b53f7-e34f-450c-ba69-ca8326d16196 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:38:01.032387 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957b53f7-e34f-450c-ba69-ca8326d16196 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:38:01.032387 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957b53f7-e34f-450c-ba69-ca8326d16196 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:38:01.034110 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562cfc60-bb61-404e-aeca-55d255147ced None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:38:01.034455 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562cfc60-bb61-404e-aeca-55d255147ced None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:38:01.034737 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562cfc60-bb61-404e-aeca-55d255147ced None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:38:01.047279 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:38:01.062710 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-957b53f7-e34f-450c-ba69-ca8326d16196 None None] MFA Rules not processed for user `2f1102c965074ac2b69809af842e3e94`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:38:01.064024 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-957b53f7-e34f-450c-ba69-ca8326d16196 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:38:01.081674 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-562cfc60-bb61-404e-aeca-55d255147ced None None] MFA Rules not processed for user `5249a2ddfc1a488994bc8c220f14db4d`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:38:01.083121 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-562cfc60-bb61-404e-aeca-55d255147ced None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:38:01.167914 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1109/2227] 10.222.0.30 () {58 vars in 958 bytes} [Tue Sep 6 04:38:01 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 04:38:01.174661 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1119/2228] 10.222.0.30 () {58 vars in 958 bytes} [Tue Sep 6 04:38:01 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 04:38:01.306154 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562cfc60-bb61-404e-aeca-55d255147ced None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:38:01.307697 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-562cfc60-bb61-404e-aeca-55d255147ced None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:38:01.370830 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562cfc60-bb61-404e-aeca-55d255147ced None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:38:01.375094 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70beabd9-2d5e-4c34-9c04-c8e413a27fce service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd09b5ab675634a2eb4b03755743f7d0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b707733ce65c42edbc1fd95ad0a1c06c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:38:01.375864 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70beabd9-2d5e-4c34-9c04-c8e413a27fce service cinder] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:38:01.379439 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70beabd9-2d5e-4c34-9c04-c8e413a27fce service cinder] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:38:01.379439 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70beabd9-2d5e-4c34-9c04-c8e413a27fce service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:38:01.393234 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70beabd9-2d5e-4c34-9c04-c8e413a27fce service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:38:01.394765 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70beabd9-2d5e-4c34-9c04-c8e413a27fce service cinder] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:38:01.406127 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1110/2229] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Sep 6 04:38:01 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 04:38:01.458265 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957b53f7-e34f-450c-ba69-ca8326d16196 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:38:01.468450 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957b53f7-e34f-450c-ba69-ca8326d16196 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:38:01.471911 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3553615a-8f58-439f-8ce2-934994c26d96 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd09b5ab675634a2eb4b03755743f7d0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b707733ce65c42edbc1fd95ad0a1c06c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:38:01.474043 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3553615a-8f58-439f-8ce2-934994c26d96 service cinder] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:38:01.474043 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3553615a-8f58-439f-8ce2-934994c26d96 service cinder] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:38:01.474043 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3553615a-8f58-439f-8ce2-934994c26d96 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:38:01.475240 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-3553615a-8f58-439f-8ce2-934994c26d96 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:38:01.537098 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3553615a-8f58-439f-8ce2-934994c26d96 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:38:01.539022 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3553615a-8f58-439f-8ce2-934994c26d96 service cinder] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:38:01.549542 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1120/2230] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Sep 6 04:38:01 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 04:38:01.676935 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70beabd9-2d5e-4c34-9c04-c8e413a27fce service cinder] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:38:01.693376 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70beabd9-2d5e-4c34-9c04-c8e413a27fce service cinder] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:38:01.697251 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39b9c56-e9f7-474f-8f7c-566e0e719c97 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd09b5ab675634a2eb4b03755743f7d0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b707733ce65c42edbc1fd95ad0a1c06c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:38:01.698099 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39b9c56-e9f7-474f-8f7c-566e0e719c97 service cinder] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:38:01.698595 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39b9c56-e9f7-474f-8f7c-566e0e719c97 service cinder] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:38:01.698595 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39b9c56-e9f7-474f-8f7c-566e0e719c97 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:38:01.760995 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c39b9c56-e9f7-474f-8f7c-566e0e719c97 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:38:01.763033 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c39b9c56-e9f7-474f-8f7c-566e0e719c97 service cinder] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:38:01.772360 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1111/2231] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Sep 6 04:38:01 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 04:38:04.758589 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3553615a-8f58-439f-8ce2-934994c26d96 service cinder] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:38:04.767647 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3553615a-8f58-439f-8ce2-934994c26d96 service cinder] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:38:04.775717 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e849989-4289-4f6a-ab67-cc98c43474bd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd09b5ab675634a2eb4b03755743f7d0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b707733ce65c42edbc1fd95ad0a1c06c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:38:04.776756 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e849989-4289-4f6a-ab67-cc98c43474bd service cinder] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:38:04.777218 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e849989-4289-4f6a-ab67-cc98c43474bd service cinder] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:38:04.777550 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e849989-4289-4f6a-ab67-cc98c43474bd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:38:04.811245 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:38:04.842114 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e849989-4289-4f6a-ab67-cc98c43474bd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:38:04.843742 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e849989-4289-4f6a-ab67-cc98c43474bd service cinder] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:38:04.853277 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1121/2232] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Sep 6 04:38:04 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 06 04:38:05.704949 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d96904c-27be-4fe6-b4f0-857e27720f47 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:38:05.705479 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d96904c-27be-4fe6-b4f0-857e27720f47 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:38:05.705571 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d96904c-27be-4fe6-b4f0-857e27720f47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:38:05.731251 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-2d96904c-27be-4fe6-b4f0-857e27720f47 None None] MFA Rules not processed for user `dec1c17550b849dbadbc16e98530b455`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:38:05.732899 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-2d96904c-27be-4fe6-b4f0-857e27720f47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:38:05.760293 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:38:05.793158 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1112/2233] 10.222.0.30 () {58 vars in 958 bytes} [Tue Sep 6 04:38:05 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 04:38:06.402752 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e849989-4289-4f6a-ab67-cc98c43474bd service cinder] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:38:06.410020 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e849989-4289-4f6a-ab67-cc98c43474bd service cinder] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:38:06.413778 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2780655-0b97-4eb7-a653-dde60bf5dbe7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd09b5ab675634a2eb4b03755743f7d0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b707733ce65c42edbc1fd95ad0a1c06c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:38:06.414383 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2780655-0b97-4eb7-a653-dde60bf5dbe7 service cinder] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:38:06.414664 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2780655-0b97-4eb7-a653-dde60bf5dbe7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:38:06.414936 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2780655-0b97-4eb7-a653-dde60bf5dbe7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:38:06.428350 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2780655-0b97-4eb7-a653-dde60bf5dbe7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:38:06.429859 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2780655-0b97-4eb7-a653-dde60bf5dbe7 service cinder] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:38:06.441047 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1122/2234] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Sep 6 04:38:06 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 04:38:09.666186 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d96904c-27be-4fe6-b4f0-857e27720f47 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:38:09.785125 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d96904c-27be-4fe6-b4f0-857e27720f47 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:38:09.790651 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837ab8bf-af19-4f62-93ad-3d5327a07a5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:38:09.792269 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837ab8bf-af19-4f62-93ad-3d5327a07a5f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:38:09.792841 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837ab8bf-af19-4f62-93ad-3d5327a07a5f admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:38:09.793336 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837ab8bf-af19-4f62-93ad-3d5327a07a5f admin admin] PATH_INFO: `/v3/users/fc58e2aa72f342f5ad66f60c2bb80926` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:38:09.794796 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837ab8bf-af19-4f62-93ad-3d5327a07a5f admin admin] RBAC: Authorizing `identity:delete_user(user_id=fc58e2aa72f342f5ad66f60c2bb80926)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:38:09.797275 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837ab8bf-af19-4f62-93ad-3d5327a07a5f admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:38:10.053807 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-837ab8bf-af19-4f62-93ad-3d5327a07a5f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fc58e2aa72f342f5ad66f60c2bb80926'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:38:10.170436 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-837ab8bf-af19-4f62-93ad-3d5327a07a5f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fc58e2aa72f342f5ad66f60c2bb80926'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:38:10.171120 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-837ab8bf-af19-4f62-93ad-3d5327a07a5f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fc58e2aa72f342f5ad66f60c2bb80926'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:38:10.179974 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-837ab8bf-af19-4f62-93ad-3d5327a07a5f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fc58e2aa72f342f5ad66f60c2bb80926'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:38:10.194701 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-837ab8bf-af19-4f62-93ad-3d5327a07a5f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fc58e2aa72f342f5ad66f60c2bb80926'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:38:10.205105 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1113/2235] 10.222.0.30 () {58 vars in 1224 bytes} [Tue Sep 6 04:38:09 2022] DELETE /identity/v3/users/fc58e2aa72f342f5ad66f60c2bb80926 => generated 0 bytes in 538 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 04:38:10.206789 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e353895b-cd19-4eff-abca-5b1163a3b018 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:38:10.207266 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e353895b-cd19-4eff-abca-5b1163a3b018 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:38:10.207524 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e353895b-cd19-4eff-abca-5b1163a3b018 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:38:10.228137 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-e353895b-cd19-4eff-abca-5b1163a3b018 None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:38:10.230624 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-e353895b-cd19-4eff-abca-5b1163a3b018 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:38:10.300295 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1123/2236] 10.222.0.30 () {58 vars in 958 bytes} [Tue Sep 6 04:38:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 04:38:10.316520 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837ab8bf-af19-4f62-93ad-3d5327a07a5f admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:38:10.318074 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-837ab8bf-af19-4f62-93ad-3d5327a07a5f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:38:10.387633 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837ab8bf-af19-4f62-93ad-3d5327a07a5f admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:38:10.391420 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ad228f-3449-453c-923a-5ddcc6b8002f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90492440f6a6471e8392361c098428aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b707733ce65c42edbc1fd95ad0a1c06c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:38:10.392167 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ad228f-3449-453c-923a-5ddcc6b8002f service neutron] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:38:10.392608 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ad228f-3449-453c-923a-5ddcc6b8002f service neutron] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:38:10.392881 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ad228f-3449-453c-923a-5ddcc6b8002f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:38:10.404530 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ad228f-3449-453c-923a-5ddcc6b8002f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:38:10.406085 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ad228f-3449-453c-923a-5ddcc6b8002f service neutron] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:38:10.410654 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:38:10.418321 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1114/2237] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Sep 6 04:38:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 04:38:10.939009 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e353895b-cd19-4eff-abca-5b1163a3b018 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:38:10.941098 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-e353895b-cd19-4eff-abca-5b1163a3b018 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:38:10.956659 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e353895b-cd19-4eff-abca-5b1163a3b018 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:38:10.966806 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6405ac95-d980-4bb7-a337-55a393f82bad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:38:10.968371 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6405ac95-d980-4bb7-a337-55a393f82bad admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:38:10.968371 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6405ac95-d980-4bb7-a337-55a393f82bad admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:38:10.968371 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6405ac95-d980-4bb7-a337-55a393f82bad admin admin] PATH_INFO: `/v3/projects/1b042cdece0a4ea3b9b9aa9c064218a8` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:38:10.969997 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6405ac95-d980-4bb7-a337-55a393f82bad admin admin] RBAC: Authorizing `identity:delete_project(project_id=1b042cdece0a4ea3b9b9aa9c064218a8)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:38:10.972247 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6405ac95-d980-4bb7-a337-55a393f82bad admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:38:11.097764 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-6405ac95-d980-4bb7-a337-55a393f82bad admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1b042cdece0a4ea3b9b9aa9c064218a8', 'request_id': 'req-6405ac95-d980-4bb7-a337-55a393f82bad'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:38:11.107930 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-6405ac95-d980-4bb7-a337-55a393f82bad admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1b042cdece0a4ea3b9b9aa9c064218a8', 'request_id': 'req-6405ac95-d980-4bb7-a337-55a393f82bad'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:38:11.164674 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-6405ac95-d980-4bb7-a337-55a393f82bad admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1b042cdece0a4ea3b9b9aa9c064218a8', 'request_id': 'req-6405ac95-d980-4bb7-a337-55a393f82bad'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:38:11.178838 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-6405ac95-d980-4bb7-a337-55a393f82bad admin admin] The token cache is being invalidate because project 1b042cdece0a4ea3b9b9aa9c064218a8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81698) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 04:38:11.179089 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-6405ac95-d980-4bb7-a337-55a393f82bad admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:38:11.179726 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-6405ac95-d980-4bb7-a337-55a393f82bad admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:38:11.181757 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1124/2238] 10.222.0.30 () {58 vars in 1233 bytes} [Tue Sep 6 04:38:10 2022] DELETE /identity/v3/projects/1b042cdece0a4ea3b9b9aa9c064218a8 => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 04:38:11.186697 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ad228f-3449-453c-923a-5ddcc6b8002f service neutron] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:38:11.200675 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ad228f-3449-453c-923a-5ddcc6b8002f service neutron] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:38:11.272136 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e108a04-985e-44e3-9da8-837c7c683f90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:38:11.272789 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e108a04-985e-44e3-9da8-837c7c683f90 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:38:11.273070 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e108a04-985e-44e3-9da8-837c7c683f90 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:38:11.273370 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e108a04-985e-44e3-9da8-837c7c683f90 admin admin] PATH_INFO: `/v3/users/5249a2ddfc1a488994bc8c220f14db4d` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:38:11.275023 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e108a04-985e-44e3-9da8-837c7c683f90 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5249a2ddfc1a488994bc8c220f14db4d)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:38:11.278121 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e108a04-985e-44e3-9da8-837c7c683f90 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:38:11.506290 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-3e108a04-985e-44e3-9da8-837c7c683f90 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5249a2ddfc1a488994bc8c220f14db4d'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:38:11.576316 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-3e108a04-985e-44e3-9da8-837c7c683f90 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5249a2ddfc1a488994bc8c220f14db4d'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:38:11.577123 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-3e108a04-985e-44e3-9da8-837c7c683f90 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5249a2ddfc1a488994bc8c220f14db4d'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:38:11.586346 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-3e108a04-985e-44e3-9da8-837c7c683f90 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5249a2ddfc1a488994bc8c220f14db4d'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:38:11.611410 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-3e108a04-985e-44e3-9da8-837c7c683f90 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5249a2ddfc1a488994bc8c220f14db4d'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:38:11.619009 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1115/2239] 10.222.0.30 () {58 vars in 1224 bytes} [Tue Sep 6 04:38:11 2022] DELETE /identity/v3/users/5249a2ddfc1a488994bc8c220f14db4d => generated 0 bytes in 433 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 04:38:12.167231 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6405ac95-d980-4bb7-a337-55a393f82bad admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:38:12.171076 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-6405ac95-d980-4bb7-a337-55a393f82bad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:38:12.183328 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6405ac95-d980-4bb7-a337-55a393f82bad admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:38:12.244291 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:38:12.263988 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d2632b-3ff9-4468-9b58-de5aa21b7d3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:38:12.264582 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d2632b-3ff9-4468-9b58-de5aa21b7d3d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:38:12.264866 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d2632b-3ff9-4468-9b58-de5aa21b7d3d admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:38:12.265140 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d2632b-3ff9-4468-9b58-de5aa21b7d3d admin admin] PATH_INFO: `/v3/projects/5d89bb133bec4b0ca1f716c742142507` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:38:12.266443 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d2632b-3ff9-4468-9b58-de5aa21b7d3d admin admin] RBAC: Authorizing `identity:delete_project(project_id=5d89bb133bec4b0ca1f716c742142507)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:38:12.268350 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d2632b-3ff9-4468-9b58-de5aa21b7d3d admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:38:12.412523 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-b5d2632b-3ff9-4468-9b58-de5aa21b7d3d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5d89bb133bec4b0ca1f716c742142507', 'request_id': 'req-b5d2632b-3ff9-4468-9b58-de5aa21b7d3d'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:38:12.423775 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-b5d2632b-3ff9-4468-9b58-de5aa21b7d3d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5d89bb133bec4b0ca1f716c742142507', 'request_id': 'req-b5d2632b-3ff9-4468-9b58-de5aa21b7d3d'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:38:12.490463 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-b5d2632b-3ff9-4468-9b58-de5aa21b7d3d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5d89bb133bec4b0ca1f716c742142507', 'request_id': 'req-b5d2632b-3ff9-4468-9b58-de5aa21b7d3d'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:38:12.503991 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-b5d2632b-3ff9-4468-9b58-de5aa21b7d3d admin admin] The token cache is being invalidate because project 5d89bb133bec4b0ca1f716c742142507 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81698) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 04:38:12.504282 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-b5d2632b-3ff9-4468-9b58-de5aa21b7d3d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:38:12.504975 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-b5d2632b-3ff9-4468-9b58-de5aa21b7d3d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:38:12.506778 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1125/2240] 10.222.0.30 () {58 vars in 1233 bytes} [Tue Sep 6 04:38:12 2022] DELETE /identity/v3/projects/5d89bb133bec4b0ca1f716c742142507 => generated 0 bytes in 340 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 04:38:12.512272 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc784b6-46de-49f7-a236-5097b1bb5adb None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:38:12.512814 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc784b6-46de-49f7-a236-5097b1bb5adb None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:38:12.512936 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc784b6-46de-49f7-a236-5097b1bb5adb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:38:12.534191 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-acc784b6-46de-49f7-a236-5097b1bb5adb None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:38:12.535335 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-acc784b6-46de-49f7-a236-5097b1bb5adb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:38:12.602834 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1116/2241] 10.222.0.30 () {58 vars in 958 bytes} [Tue Sep 6 04:38:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 04:38:12.611207 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0a3e3d-44ee-4fc4-8bdb-35043932d623 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:38:12.611640 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0a3e3d-44ee-4fc4-8bdb-35043932d623 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:38:12.611994 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0a3e3d-44ee-4fc4-8bdb-35043932d623 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:38:12.635821 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-af0a3e3d-44ee-4fc4-8bdb-35043932d623 None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:38:12.637606 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-af0a3e3d-44ee-4fc4-8bdb-35043932d623 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:38:12.646454 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1126/2242] 10.222.0.30 () {58 vars in 958 bytes} [Tue Sep 6 04:38:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 04:38:12.651556 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc784b6-46de-49f7-a236-5097b1bb5adb None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:38:12.663613 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc784b6-46de-49f7-a236-5097b1bb5adb None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:38:12.669104 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a987993-1666-45cc-9a33-28eb244637eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:38:12.669944 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a987993-1666-45cc-9a33-28eb244637eb admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:38:12.670221 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a987993-1666-45cc-9a33-28eb244637eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:38:12.670491 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a987993-1666-45cc-9a33-28eb244637eb admin admin] PATH_INFO: `/v3/domains` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:38:12.671156 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a987993-1666-45cc-9a33-28eb244637eb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:38:12.671460 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a987993-1666-45cc-9a33-28eb244637eb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:38:12.673070 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a987993-1666-45cc-9a33-28eb244637eb admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:38:12.690050 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1117/2243] 10.222.0.30 () {58 vars in 1153 bytes} [Tue Sep 6 04:38:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:38:12.691117 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0a3e3d-44ee-4fc4-8bdb-35043932d623 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:38:12.702971 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0a3e3d-44ee-4fc4-8bdb-35043932d623 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:38:12.711739 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d92bcb9-63ea-403f-a46a-908b631abe1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:38:12.712396 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d92bcb9-63ea-403f-a46a-908b631abe1e admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:38:12.712673 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d92bcb9-63ea-403f-a46a-908b631abe1e admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:38:12.713032 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d92bcb9-63ea-403f-a46a-908b631abe1e admin admin] PATH_INFO: `/v3/projects` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:38:12.714222 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d92bcb9-63ea-403f-a46a-908b631abe1e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:38:12.716246 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d92bcb9-63ea-403f-a46a-908b631abe1e admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:38:12.863334 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1127/2244] 10.222.0.30 () {60 vars in 1153 bytes} [Tue Sep 6 04:38:12 2022] POST /identity/v3/projects => generated 375 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 04:38:12.867706 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a987993-1666-45cc-9a33-28eb244637eb admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:38:12.875235 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:38:12.879053 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a987993-1666-45cc-9a33-28eb244637eb admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:38:12.940876 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5475cbb-273c-4074-80b6-38b89e7a5526 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:38:12.941571 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5475cbb-273c-4074-80b6-38b89e7a5526 admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:38:12.942118 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5475cbb-273c-4074-80b6-38b89e7a5526 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:38:12.942432 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5475cbb-273c-4074-80b6-38b89e7a5526 admin admin] PATH_INFO: `/v3/users` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:38:12.943072 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5475cbb-273c-4074-80b6-38b89e7a5526 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:38:12.944987 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5475cbb-273c-4074-80b6-38b89e7a5526 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:38:13.039285 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1118/2245] 10.222.0.30 () {60 vars in 1144 bytes} [Tue Sep 6 04:38:12 2022] POST /identity/v3/users => generated 288 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 04:38:13.044769 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d92bcb9-63ea-403f-a46a-908b631abe1e admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:38:13.055754 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d92bcb9-63ea-403f-a46a-908b631abe1e admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:38:13.060844 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d4b16d-73cd-4746-8084-a11e03d6670c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:38:13.061503 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d4b16d-73cd-4746-8084-a11e03d6670c admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:38:13.061784 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d4b16d-73cd-4746-8084-a11e03d6670c admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:38:13.062053 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d4b16d-73cd-4746-8084-a11e03d6670c admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:38:13.062741 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d4b16d-73cd-4746-8084-a11e03d6670c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:38:13.064558 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d4b16d-73cd-4746-8084-a11e03d6670c admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:38:13.076246 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1128/2246] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Sep 6 04:38:13 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 04:38:13.080374 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5475cbb-273c-4074-80b6-38b89e7a5526 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:38:13.091164 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5475cbb-273c-4074-80b6-38b89e7a5526 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:38:13.096759 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e9cf3e-85fe-402f-8a21-b4f08e6227a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:38:13.101949 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e9cf3e-85fe-402f-8a21-b4f08e6227a1 admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:38:13.101949 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e9cf3e-85fe-402f-8a21-b4f08e6227a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:38:13.101949 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e9cf3e-85fe-402f-8a21-b4f08e6227a1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:38:13.101949 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e9cf3e-85fe-402f-8a21-b4f08e6227a1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:38:13.101949 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e9cf3e-85fe-402f-8a21-b4f08e6227a1 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:38:13.111884 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1119/2247] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Sep 6 04:38:13 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 06 04:38:13.116035 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d4b16d-73cd-4746-8084-a11e03d6670c admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:38:13.126637 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d4b16d-73cd-4746-8084-a11e03d6670c admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:38:13.131247 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb14f2b4-7d66-418b-87dd-63784505c92a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:38:13.131887 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb14f2b4-7d66-418b-87dd-63784505c92a admin admin] REQUEST_METHOD: `PUT` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:38:13.132162 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb14f2b4-7d66-418b-87dd-63784505c92a admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:38:13.132431 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb14f2b4-7d66-418b-87dd-63784505c92a admin admin] PATH_INFO: `/v3/projects/7e953380840c42f7b7904aa0dc5638fc/users/8eb559b8c3d34d3e86b7d52dffd4dccb/roles/29607b2de39f40d68bb6ceef64b8a732` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:38:13.153728 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb14f2b4-7d66-418b-87dd-63784505c92a admin admin] RBAC: Authorizing `identity:create_grant(project_id=7e953380840c42f7b7904aa0dc5638fc, user_id=8eb559b8c3d34d3e86b7d52dffd4dccb, role_id=29607b2de39f40d68bb6ceef64b8a732)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:38:13.156362 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb14f2b4-7d66-418b-87dd-63784505c92a admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:38:13.251769 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1129/2248] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Sep 6 04:38:13 2022] PUT /identity/v3/projects/7e953380840c42f7b7904aa0dc5638fc/users/8eb559b8c3d34d3e86b7d52dffd4dccb/roles/29607b2de39f40d68bb6ceef64b8a732 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 04:38:13.263438 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e551d921-1026-4bbf-936f-d0767bf82edc None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:38:13.263726 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e551d921-1026-4bbf-936f-d0767bf82edc None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:38:13.264032 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e551d921-1026-4bbf-936f-d0767bf82edc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:38:13.292829 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-e551d921-1026-4bbf-936f-d0767bf82edc None None] MFA Rules not processed for user `8eb559b8c3d34d3e86b7d52dffd4dccb`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:38:13.294089 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-e551d921-1026-4bbf-936f-d0767bf82edc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:38:13.372321 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1120/2249] 10.222.0.30 () {58 vars in 958 bytes} [Tue Sep 6 04:38:13 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 04:38:13.376798 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb14f2b4-7d66-418b-87dd-63784505c92a admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:38:13.439179 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb14f2b4-7d66-418b-87dd-63784505c92a admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:38:13.443938 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9e99c8-7e21-431b-a02c-e09924472f71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:38:13.444545 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9e99c8-7e21-431b-a02c-e09924472f71 admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:38:13.444827 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9e99c8-7e21-431b-a02c-e09924472f71 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:38:13.445102 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9e99c8-7e21-431b-a02c-e09924472f71 admin admin] PATH_INFO: `/v3/projects` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:38:13.445798 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9e99c8-7e21-431b-a02c-e09924472f71 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:38:13.447720 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9e99c8-7e21-431b-a02c-e09924472f71 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:38:13.459997 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:38:13.526610 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e551d921-1026-4bbf-936f-d0767bf82edc None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:38:13.528063 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-e551d921-1026-4bbf-936f-d0767bf82edc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:38:13.541377 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e551d921-1026-4bbf-936f-d0767bf82edc None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:38:13.546643 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d0f9ed-eb93-475e-bb12-ed492096a524 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:38:13.547338 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d0f9ed-eb93-475e-bb12-ed492096a524 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:38:13.547500 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d0f9ed-eb93-475e-bb12-ed492096a524 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:38:13.547770 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d0f9ed-eb93-475e-bb12-ed492096a524 admin admin] PATH_INFO: `/v3/users/93ad8fce80e846abaec07eeafc74b065` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:38:13.548881 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d0f9ed-eb93-475e-bb12-ed492096a524 admin admin] RBAC: Authorizing `identity:delete_user(user_id=93ad8fce80e846abaec07eeafc74b065)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:38:13.550886 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d0f9ed-eb93-475e-bb12-ed492096a524 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:38:13.575989 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1130/2250] 10.222.0.30 () {60 vars in 1153 bytes} [Tue Sep 6 04:38:13 2022] POST /identity/v3/projects => generated 377 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 04:38:13.575989 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9e99c8-7e21-431b-a02c-e09924472f71 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:38:13.635909 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9e99c8-7e21-431b-a02c-e09924472f71 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:38:13.641354 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fac018-4ace-4254-af3c-368bfe203f44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:38:13.641961 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fac018-4ace-4254-af3c-368bfe203f44 admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:38:13.642243 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fac018-4ace-4254-af3c-368bfe203f44 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:38:13.642526 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fac018-4ace-4254-af3c-368bfe203f44 admin admin] PATH_INFO: `/v3/users` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:38:13.643142 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9fac018-4ace-4254-af3c-368bfe203f44 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:38:13.645054 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9fac018-4ace-4254-af3c-368bfe203f44 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:38:13.747575 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1131/2251] 10.222.0.30 () {60 vars in 1144 bytes} [Tue Sep 6 04:38:13 2022] POST /identity/v3/users => generated 289 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 04:38:13.751506 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fac018-4ace-4254-af3c-368bfe203f44 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:38:13.761125 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fac018-4ace-4254-af3c-368bfe203f44 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:38:13.766812 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f75bc61-dfe8-47f6-a70c-a2a98ea3426b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:38:13.767410 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f75bc61-dfe8-47f6-a70c-a2a98ea3426b admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:38:13.767549 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-c8d0f9ed-eb93-475e-bb12-ed492096a524 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '93ad8fce80e846abaec07eeafc74b065'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:38:13.767685 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f75bc61-dfe8-47f6-a70c-a2a98ea3426b admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:38:13.767950 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f75bc61-dfe8-47f6-a70c-a2a98ea3426b admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:38:13.768590 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f75bc61-dfe8-47f6-a70c-a2a98ea3426b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:38:13.770279 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f75bc61-dfe8-47f6-a70c-a2a98ea3426b admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:38:13.781600 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1132/2252] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Sep 6 04:38:13 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 06 04:38:13.785473 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f75bc61-dfe8-47f6-a70c-a2a98ea3426b admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:38:13.794969 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f75bc61-dfe8-47f6-a70c-a2a98ea3426b admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:38:13.799787 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731fc440-370f-48e3-a769-c12651e8af44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:38:13.800352 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731fc440-370f-48e3-a769-c12651e8af44 admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:38:13.800629 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731fc440-370f-48e3-a769-c12651e8af44 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:38:13.800897 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731fc440-370f-48e3-a769-c12651e8af44 admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:38:13.801563 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731fc440-370f-48e3-a769-c12651e8af44 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:38:13.803112 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731fc440-370f-48e3-a769-c12651e8af44 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:38:13.813494 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1133/2253] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Sep 6 04:38:13 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 06 04:38:13.819973 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731fc440-370f-48e3-a769-c12651e8af44 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:38:13.827841 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-c8d0f9ed-eb93-475e-bb12-ed492096a524 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '93ad8fce80e846abaec07eeafc74b065'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:38:13.829097 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-c8d0f9ed-eb93-475e-bb12-ed492096a524 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '93ad8fce80e846abaec07eeafc74b065'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:38:13.830951 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731fc440-370f-48e3-a769-c12651e8af44 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:38:13.836165 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbdcd012-7b5d-47ea-b987-3b86969c29cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:38:13.837513 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdcd012-7b5d-47ea-b987-3b86969c29cf admin admin] REQUEST_METHOD: `PUT` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:38:13.837806 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdcd012-7b5d-47ea-b987-3b86969c29cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:38:13.839015 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-c8d0f9ed-eb93-475e-bb12-ed492096a524 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '93ad8fce80e846abaec07eeafc74b065'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:38:13.839579 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdcd012-7b5d-47ea-b987-3b86969c29cf admin admin] PATH_INFO: `/v3/projects/b7bb7cded1a64fc4a3128e7abc625472/users/bdc674a9b6a04d84abbbcba4c16cc4b1/roles/29607b2de39f40d68bb6ceef64b8a732` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:38:13.845967 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=88,keystone:INSERT=4,keystone:DELETE=8 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:38:13.854583 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-c8d0f9ed-eb93-475e-bb12-ed492096a524 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '93ad8fce80e846abaec07eeafc74b065'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:38:13.858006 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbdcd012-7b5d-47ea-b987-3b86969c29cf admin admin] RBAC: Authorizing `identity:create_grant(project_id=b7bb7cded1a64fc4a3128e7abc625472, user_id=bdc674a9b6a04d84abbbcba4c16cc4b1, role_id=29607b2de39f40d68bb6ceef64b8a732)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:38:13.861378 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1121/2254] 10.222.0.30 () {58 vars in 1224 bytes} [Tue Sep 6 04:38:13 2022] DELETE /identity/v3/users/93ad8fce80e846abaec07eeafc74b065 => generated 0 bytes in 335 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 04:38:13.861884 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbdcd012-7b5d-47ea-b987-3b86969c29cf admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:38:13.865518 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9be130-f73f-408b-ae32-6a97ddafa4e1 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:38:13.865808 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9be130-f73f-408b-ae32-6a97ddafa4e1 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:38:13.866087 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9be130-f73f-408b-ae32-6a97ddafa4e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:38:13.885018 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-be9be130-f73f-408b-ae32-6a97ddafa4e1 None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:38:13.886289 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-be9be130-f73f-408b-ae32-6a97ddafa4e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:38:13.943628 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1134/2255] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Sep 6 04:38:13 2022] PUT /identity/v3/projects/b7bb7cded1a64fc4a3128e7abc625472/users/bdc674a9b6a04d84abbbcba4c16cc4b1/roles/29607b2de39f40d68bb6ceef64b8a732 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 04:38:13.955891 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d3fc6f-3d1c-4fdd-b6b3-cb99d43215f9 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:38:13.956179 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d3fc6f-3d1c-4fdd-b6b3-cb99d43215f9 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:38:13.956583 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d3fc6f-3d1c-4fdd-b6b3-cb99d43215f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:38:13.969679 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1122/2256] 10.222.0.30 () {58 vars in 958 bytes} [Tue Sep 6 04:38:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 04:38:13.980591 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9be130-f73f-408b-ae32-6a97ddafa4e1 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:38:13.982497 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-be9be130-f73f-408b-ae32-6a97ddafa4e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:38:13.991320 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-f0d3fc6f-3d1c-4fdd-b6b3-cb99d43215f9 None None] MFA Rules not processed for user `bdc674a9b6a04d84abbbcba4c16cc4b1`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:38:13.992613 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-f0d3fc6f-3d1c-4fdd-b6b3-cb99d43215f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:38:14.047040 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9be130-f73f-408b-ae32-6a97ddafa4e1 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:38:14.071910 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1135/2257] 10.222.0.30 () {58 vars in 958 bytes} [Tue Sep 6 04:38:13 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 04:38:14.077095 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d3fc6f-3d1c-4fdd-b6b3-cb99d43215f9 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:38:14.078952 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-f0d3fc6f-3d1c-4fdd-b6b3-cb99d43215f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:38:14.100115 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d759b61-f249-4790-86a0-9e84b710fbf6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90492440f6a6471e8392361c098428aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b707733ce65c42edbc1fd95ad0a1c06c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:38:14.100758 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d759b61-f249-4790-86a0-9e84b710fbf6 service neutron] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:38:14.101041 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d759b61-f249-4790-86a0-9e84b710fbf6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:38:14.101343 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d759b61-f249-4790-86a0-9e84b710fbf6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:38:14.107420 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:38:14.145605 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d3fc6f-3d1c-4fdd-b6b3-cb99d43215f9 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:38:14.150749 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b329dc54-680f-4d3d-8db5-66dbcbb9cd82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:38:14.151528 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b329dc54-680f-4d3d-8db5-66dbcbb9cd82 admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:38:14.151815 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b329dc54-680f-4d3d-8db5-66dbcbb9cd82 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:38:14.152277 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b329dc54-680f-4d3d-8db5-66dbcbb9cd82 admin admin] PATH_INFO: `/v3/projects` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:38:14.152924 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b329dc54-680f-4d3d-8db5-66dbcbb9cd82 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:38:14.153085 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:DELETE=2,keystone:SELECT=98 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:38:14.154912 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b329dc54-680f-4d3d-8db5-66dbcbb9cd82 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:38:14.162895 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d759b61-f249-4790-86a0-9e84b710fbf6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:38:14.164378 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d759b61-f249-4790-86a0-9e84b710fbf6 service neutron] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:38:14.173993 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1123/2258] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Sep 6 04:38:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 195 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 04:38:14.260787 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1136/2259] 10.222.0.30 () {60 vars in 1153 bytes} [Tue Sep 6 04:38:14 2022] POST /identity/v3/projects => generated 375 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 04:38:14.262769 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d759b61-f249-4790-86a0-9e84b710fbf6 service neutron] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:38:14.272708 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d759b61-f249-4790-86a0-9e84b710fbf6 service neutron] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:38:14.330589 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1847fce4-b961-4f02-9b91-0bce2bf316c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:38:14.331216 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1847fce4-b961-4f02-9b91-0bce2bf316c2 admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:38:14.331492 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1847fce4-b961-4f02-9b91-0bce2bf316c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:38:14.331763 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1847fce4-b961-4f02-9b91-0bce2bf316c2 admin admin] PATH_INFO: `/v3/users` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:38:14.332407 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1847fce4-b961-4f02-9b91-0bce2bf316c2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:38:14.334410 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1847fce4-b961-4f02-9b91-0bce2bf316c2 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:38:14.407654 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1124/2260] 10.222.0.30 () {60 vars in 1144 bytes} [Tue Sep 6 04:38:14 2022] POST /identity/v3/users => generated 294 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 04:38:14.411935 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b329dc54-680f-4d3d-8db5-66dbcbb9cd82 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:38:14.423677 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b329dc54-680f-4d3d-8db5-66dbcbb9cd82 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:38:14.428779 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edeb5705-928f-4807-ac87-52167ca592f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:38:14.429223 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edeb5705-928f-4807-ac87-52167ca592f9 admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:38:14.429556 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edeb5705-928f-4807-ac87-52167ca592f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:38:14.429882 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edeb5705-928f-4807-ac87-52167ca592f9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:38:14.430565 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edeb5705-928f-4807-ac87-52167ca592f9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:38:14.432212 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edeb5705-928f-4807-ac87-52167ca592f9 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:38:14.445090 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1137/2261] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Sep 6 04:38:14 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 04:38:14.449547 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1847fce4-b961-4f02-9b91-0bce2bf316c2 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:38:14.461494 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1847fce4-b961-4f02-9b91-0bce2bf316c2 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:38:14.467812 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51ad5d9-6e2e-484b-94b1-d1183a9ca651 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:38:14.468502 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51ad5d9-6e2e-484b-94b1-d1183a9ca651 admin admin] REQUEST_METHOD: `PUT` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:38:14.468786 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51ad5d9-6e2e-484b-94b1-d1183a9ca651 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:38:14.469065 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51ad5d9-6e2e-484b-94b1-d1183a9ca651 admin admin] PATH_INFO: `/v3/projects/b9d0c78786d1440d9fcbf8c174fe846b/users/1462b2564e0d414591dc6f137e391d1d/roles/ea4c814713b149108e488de855b025e4` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:38:14.485503 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c51ad5d9-6e2e-484b-94b1-d1183a9ca651 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b9d0c78786d1440d9fcbf8c174fe846b, user_id=1462b2564e0d414591dc6f137e391d1d, role_id=ea4c814713b149108e488de855b025e4)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:38:14.488150 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c51ad5d9-6e2e-484b-94b1-d1183a9ca651 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:38:14.582017 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1125/2262] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Sep 6 04:38:14 2022] PUT /identity/v3/projects/b9d0c78786d1440d9fcbf8c174fe846b/users/1462b2564e0d414591dc6f137e391d1d/roles/ea4c814713b149108e488de855b025e4 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 04:38:14.585911 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edeb5705-928f-4807-ac87-52167ca592f9 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:38:14.596039 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edeb5705-928f-4807-ac87-52167ca592f9 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:38:14.612718 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51ad5d9-6e2e-484b-94b1-d1183a9ca651 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:38:14.616549 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-c51ad5d9-6e2e-484b-94b1-d1183a9ca651 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:38:14.684861 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e10b5c-95e9-4127-b171-6a8021e182e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:38:14.686887 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e10b5c-95e9-4127-b171-6a8021e182e6 admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:38:14.686887 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e10b5c-95e9-4127-b171-6a8021e182e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:38:14.686887 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e10b5c-95e9-4127-b171-6a8021e182e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:38:14.686887 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e10b5c-95e9-4127-b171-6a8021e182e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:38:14.688277 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e10b5c-95e9-4127-b171-6a8021e182e6 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:38:14.698864 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1138/2263] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Sep 6 04:38:14 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 04:38:14.703129 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e10b5c-95e9-4127-b171-6a8021e182e6 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:38:14.713105 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51ad5d9-6e2e-484b-94b1-d1183a9ca651 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:38:14.715614 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e10b5c-95e9-4127-b171-6a8021e182e6 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:38:14.718878 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bce4b0e-0d64-4ea1-bab0-86d245be0146 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:38:14.719111 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a1dd85-ddc5-4619-92ab-e3d54bf8cf41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:38:14.719536 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bce4b0e-0d64-4ea1-bab0-86d245be0146 admin admin] REQUEST_METHOD: `PUT` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:38:14.719699 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a1dd85-ddc5-4619-92ab-e3d54bf8cf41 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:38:14.719799 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bce4b0e-0d64-4ea1-bab0-86d245be0146 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:38:14.719983 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a1dd85-ddc5-4619-92ab-e3d54bf8cf41 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:38:14.720078 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bce4b0e-0d64-4ea1-bab0-86d245be0146 admin admin] PATH_INFO: `/v3/projects/b9d0c78786d1440d9fcbf8c174fe846b/users/1462b2564e0d414591dc6f137e391d1d/roles/ea4c814713b149108e488de855b025e4` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:38:14.720252 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a1dd85-ddc5-4619-92ab-e3d54bf8cf41 admin admin] PATH_INFO: `/v3/projects/2fdab554573b41e3a827b0bc99cd374b` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:38:14.721420 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a1dd85-ddc5-4619-92ab-e3d54bf8cf41 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2fdab554573b41e3a827b0bc99cd374b)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:38:14.722116 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bce4b0e-0d64-4ea1-bab0-86d245be0146 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b9d0c78786d1440d9fcbf8c174fe846b, user_id=1462b2564e0d414591dc6f137e391d1d, role_id=ea4c814713b149108e488de855b025e4)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:38:14.723314 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a1dd85-ddc5-4619-92ab-e3d54bf8cf41 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:38:14.724723 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bce4b0e-0d64-4ea1-bab0-86d245be0146 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:38:14.737343 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1139/2264] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Sep 6 04:38:14 2022] PUT /identity/v3/projects/b9d0c78786d1440d9fcbf8c174fe846b/users/1462b2564e0d414591dc6f137e391d1d/roles/ea4c814713b149108e488de855b025e4 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 04:38:14.749696 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc0fed6-855a-4613-ac02-6f8227efd2a8 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:38:14.750375 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc0fed6-855a-4613-ac02-6f8227efd2a8 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:38:14.750650 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc0fed6-855a-4613-ac02-6f8227efd2a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:38:14.780197 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-0fc0fed6-855a-4613-ac02-6f8227efd2a8 None None] MFA Rules not processed for user `1462b2564e0d414591dc6f137e391d1d`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:38:14.781578 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-0fc0fed6-855a-4613-ac02-6f8227efd2a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:38:14.846218 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-46a1dd85-ddc5-4619-92ab-e3d54bf8cf41 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2fdab554573b41e3a827b0bc99cd374b', 'request_id': 'req-46a1dd85-ddc5-4619-92ab-e3d54bf8cf41'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:38:14.850633 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1140/2265] 10.222.0.30 () {58 vars in 958 bytes} [Tue Sep 6 04:38:14 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 04:38:14.862602 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc0fed6-855a-4613-ac02-6f8227efd2a8 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:38:14.863961 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-0fc0fed6-855a-4613-ac02-6f8227efd2a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:38:14.873665 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:38:14.919908 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-46a1dd85-ddc5-4619-92ab-e3d54bf8cf41 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2fdab554573b41e3a827b0bc99cd374b', 'request_id': 'req-46a1dd85-ddc5-4619-92ab-e3d54bf8cf41'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:38:14.921395 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc0fed6-855a-4613-ac02-6f8227efd2a8 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:38:14.922792 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=90,keystone:INSERT=5,keystone:DELETE=5 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:38:14.927467 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-46a1dd85-ddc5-4619-92ab-e3d54bf8cf41 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2fdab554573b41e3a827b0bc99cd374b', 'request_id': 'req-46a1dd85-ddc5-4619-92ab-e3d54bf8cf41'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:38:14.938087 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-46a1dd85-ddc5-4619-92ab-e3d54bf8cf41 admin admin] The token cache is being invalidate because project 2fdab554573b41e3a827b0bc99cd374b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81697) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 04:38:14.938382 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-46a1dd85-ddc5-4619-92ab-e3d54bf8cf41 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:38:14.939042 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-46a1dd85-ddc5-4619-92ab-e3d54bf8cf41 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:38:14.940579 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1126/2266] 10.222.0.30 () {58 vars in 1233 bytes} [Tue Sep 6 04:38:14 2022] DELETE /identity/v3/projects/2fdab554573b41e3a827b0bc99cd374b => generated 0 bytes in 328 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 04:38:14.944471 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a1dd85-ddc5-4619-92ab-e3d54bf8cf41 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:38:14.945868 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-46a1dd85-ddc5-4619-92ab-e3d54bf8cf41 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:38:14.972810 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed10808c-2540-4439-beca-53f2a998acc4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd09b5ab675634a2eb4b03755743f7d0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b707733ce65c42edbc1fd95ad0a1c06c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:38:14.973455 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed10808c-2540-4439-beca-53f2a998acc4 service cinder] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:38:14.973739 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed10808c-2540-4439-beca-53f2a998acc4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:38:14.974015 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed10808c-2540-4439-beca-53f2a998acc4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:38:14.975905 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-ed10808c-2540-4439-beca-53f2a998acc4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:38:15.012170 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a1dd85-ddc5-4619-92ab-e3d54bf8cf41 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:38:15.017236 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178ef301-6472-4d76-a634-0bbdfcaf484a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:38:15.017913 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178ef301-6472-4d76-a634-0bbdfcaf484a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:38:15.018199 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178ef301-6472-4d76-a634-0bbdfcaf484a admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:38:15.018470 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178ef301-6472-4d76-a634-0bbdfcaf484a admin admin] PATH_INFO: `/v3/users/23e117b9d61340c9a59420cfa69d4c01` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:38:15.019680 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-178ef301-6472-4d76-a634-0bbdfcaf484a admin admin] RBAC: Authorizing `identity:delete_user(user_id=23e117b9d61340c9a59420cfa69d4c01)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:38:15.021676 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-178ef301-6472-4d76-a634-0bbdfcaf484a admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:38:15.052656 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed10808c-2540-4439-beca-53f2a998acc4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:38:15.054296 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed10808c-2540-4439-beca-53f2a998acc4 service cinder] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:38:15.065233 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1141/2267] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Sep 6 04:38:14 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 203 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 06 04:38:15.293578 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-178ef301-6472-4d76-a634-0bbdfcaf484a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '23e117b9d61340c9a59420cfa69d4c01'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:38:15.410146 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-178ef301-6472-4d76-a634-0bbdfcaf484a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '23e117b9d61340c9a59420cfa69d4c01'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:38:15.410787 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-178ef301-6472-4d76-a634-0bbdfcaf484a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '23e117b9d61340c9a59420cfa69d4c01'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:38:15.419450 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-178ef301-6472-4d76-a634-0bbdfcaf484a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '23e117b9d61340c9a59420cfa69d4c01'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:38:15.431084 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-178ef301-6472-4d76-a634-0bbdfcaf484a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '23e117b9d61340c9a59420cfa69d4c01'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:38:15.437219 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1127/2268] 10.222.0.30 () {58 vars in 1224 bytes} [Tue Sep 6 04:38:14 2022] DELETE /identity/v3/users/23e117b9d61340c9a59420cfa69d4c01 => generated 0 bytes in 493 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 04:38:15.708133 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed10808c-2540-4439-beca-53f2a998acc4 service cinder] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:38:15.710151 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-ed10808c-2540-4439-beca-53f2a998acc4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:38:15.733351 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed10808c-2540-4439-beca-53f2a998acc4 service cinder] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:38:15.801518 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d900564e-02ae-4942-9640-2023af79ecd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:38:15.802162 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d900564e-02ae-4942-9640-2023af79ecd2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:38:15.802441 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d900564e-02ae-4942-9640-2023af79ecd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:38:15.802810 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d900564e-02ae-4942-9640-2023af79ecd2 admin admin] PATH_INFO: `/v3/users/fc6eb1c7a71b41288f1ec882c45f261c` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:38:15.803944 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d900564e-02ae-4942-9640-2023af79ecd2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fc6eb1c7a71b41288f1ec882c45f261c)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:38:15.805910 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d900564e-02ae-4942-9640-2023af79ecd2 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:38:15.827607 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:38:15.841207 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178ef301-6472-4d76-a634-0bbdfcaf484a admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:38:15.843572 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-178ef301-6472-4d76-a634-0bbdfcaf484a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:38:15.859640 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178ef301-6472-4d76-a634-0bbdfcaf484a admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:38:15.865625 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781304a1-d354-474c-b30f-66ddaa51f988 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:38:15.866275 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781304a1-d354-474c-b30f-66ddaa51f988 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:38:15.866559 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781304a1-d354-474c-b30f-66ddaa51f988 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:38:15.866883 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781304a1-d354-474c-b30f-66ddaa51f988 admin admin] PATH_INFO: `/v3/projects/5f2bb0dd0dac49c6b93117c169be1ce1` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:38:15.868127 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-781304a1-d354-474c-b30f-66ddaa51f988 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5f2bb0dd0dac49c6b93117c169be1ce1)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:38:15.875125 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-781304a1-d354-474c-b30f-66ddaa51f988 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:38:16.013521 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-d900564e-02ae-4942-9640-2023af79ecd2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fc6eb1c7a71b41288f1ec882c45f261c'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:38:16.020081 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-781304a1-d354-474c-b30f-66ddaa51f988 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5f2bb0dd0dac49c6b93117c169be1ce1', 'request_id': 'req-781304a1-d354-474c-b30f-66ddaa51f988'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:38:16.084681 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-d900564e-02ae-4942-9640-2023af79ecd2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fc6eb1c7a71b41288f1ec882c45f261c'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:38:16.085328 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-d900564e-02ae-4942-9640-2023af79ecd2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fc6eb1c7a71b41288f1ec882c45f261c'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:38:16.098814 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-d900564e-02ae-4942-9640-2023af79ecd2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fc6eb1c7a71b41288f1ec882c45f261c'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:38:16.106007 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-d900564e-02ae-4942-9640-2023af79ecd2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fc6eb1c7a71b41288f1ec882c45f261c'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:38:16.112651 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1142/2269] 10.222.0.30 () {58 vars in 1224 bytes} [Tue Sep 6 04:38:15 2022] DELETE /identity/v3/users/fc6eb1c7a71b41288f1ec882c45f261c => generated 0 bytes in 405 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 04:38:16.117060 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843bdd6d-a4f0-4114-a423-365966cb9631 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:38:16.117378 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843bdd6d-a4f0-4114-a423-365966cb9631 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:38:16.117893 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843bdd6d-a4f0-4114-a423-365966cb9631 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:38:16.145178 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-843bdd6d-a4f0-4114-a423-365966cb9631 None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:38:16.145178 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-843bdd6d-a4f0-4114-a423-365966cb9631 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:38:16.147027 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-781304a1-d354-474c-b30f-66ddaa51f988 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5f2bb0dd0dac49c6b93117c169be1ce1', 'request_id': 'req-781304a1-d354-474c-b30f-66ddaa51f988'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:38:16.154170 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-781304a1-d354-474c-b30f-66ddaa51f988 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5f2bb0dd0dac49c6b93117c169be1ce1', 'request_id': 'req-781304a1-d354-474c-b30f-66ddaa51f988'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:38:16.155389 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:38:16.164954 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-781304a1-d354-474c-b30f-66ddaa51f988 admin admin] The token cache is being invalidate because project 5f2bb0dd0dac49c6b93117c169be1ce1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81697) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 04:38:16.165917 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-781304a1-d354-474c-b30f-66ddaa51f988 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:38:16.166574 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-781304a1-d354-474c-b30f-66ddaa51f988 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:38:16.168627 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1128/2270] 10.222.0.30 () {58 vars in 1233 bytes} [Tue Sep 6 04:38:15 2022] DELETE /identity/v3/projects/5f2bb0dd0dac49c6b93117c169be1ce1 => generated 0 bytes in 328 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 04:38:16.172531 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781304a1-d354-474c-b30f-66ddaa51f988 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:38:16.173924 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-781304a1-d354-474c-b30f-66ddaa51f988 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:38:16.199873 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1143/2271] 10.222.0.30 () {58 vars in 958 bytes} [Tue Sep 6 04:38:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 04:38:16.210949 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843bdd6d-a4f0-4114-a423-365966cb9631 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:38:16.212369 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-843bdd6d-a4f0-4114-a423-365966cb9631 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:38:16.247736 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781304a1-d354-474c-b30f-66ddaa51f988 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:38:16.252973 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4167acbe-1966-4be7-afa4-ff743ec3b697 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:38:16.253675 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4167acbe-1966-4be7-afa4-ff743ec3b697 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:38:16.253954 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4167acbe-1966-4be7-afa4-ff743ec3b697 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:38:16.254225 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4167acbe-1966-4be7-afa4-ff743ec3b697 admin admin] PATH_INFO: `/v3/users/2f1102c965074ac2b69809af842e3e94` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:38:16.255487 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4167acbe-1966-4be7-afa4-ff743ec3b697 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2f1102c965074ac2b69809af842e3e94)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:38:16.257456 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4167acbe-1966-4be7-afa4-ff743ec3b697 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:38:16.289044 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843bdd6d-a4f0-4114-a423-365966cb9631 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:38:16.293589 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f786f8-37a3-4cea-9047-b7bf3a9a028f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90492440f6a6471e8392361c098428aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b707733ce65c42edbc1fd95ad0a1c06c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:38:16.294475 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f786f8-37a3-4cea-9047-b7bf3a9a028f service neutron] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:38:16.294762 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f786f8-37a3-4cea-9047-b7bf3a9a028f service neutron] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:38:16.295027 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f786f8-37a3-4cea-9047-b7bf3a9a028f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:38:16.307618 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f786f8-37a3-4cea-9047-b7bf3a9a028f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:38:16.309481 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f786f8-37a3-4cea-9047-b7bf3a9a028f service neutron] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:38:16.323427 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1144/2272] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Sep 6 04:38:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 06 04:38:16.419600 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-4167acbe-1966-4be7-afa4-ff743ec3b697 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2f1102c965074ac2b69809af842e3e94'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:38:16.479040 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-4167acbe-1966-4be7-afa4-ff743ec3b697 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2f1102c965074ac2b69809af842e3e94'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:38:16.479662 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-4167acbe-1966-4be7-afa4-ff743ec3b697 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2f1102c965074ac2b69809af842e3e94'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:38:16.487339 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-4167acbe-1966-4be7-afa4-ff743ec3b697 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2f1102c965074ac2b69809af842e3e94'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:38:16.499432 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-4167acbe-1966-4be7-afa4-ff743ec3b697 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2f1102c965074ac2b69809af842e3e94'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:38:16.505425 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1129/2273] 10.222.0.30 () {58 vars in 1224 bytes} [Tue Sep 6 04:38:16 2022] DELETE /identity/v3/users/2f1102c965074ac2b69809af842e3e94 => generated 0 bytes in 333 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 04:38:16.877889 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f786f8-37a3-4cea-9047-b7bf3a9a028f service neutron] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:38:16.879379 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-b3f786f8-37a3-4cea-9047-b7bf3a9a028f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:38:16.892738 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f786f8-37a3-4cea-9047-b7bf3a9a028f service neutron] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:38:16.933753 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:DELETE=8,keystone:SELECT=91,keystone:INSERT=1 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:38:16.942924 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4167acbe-1966-4be7-afa4-ff743ec3b697 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:38:16.944463 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-4167acbe-1966-4be7-afa4-ff743ec3b697 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:38:16.957589 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc3ce63-fde2-4297-af96-111c8ab4838b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:38:16.958310 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc3ce63-fde2-4297-af96-111c8ab4838b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:38:16.958665 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc3ce63-fde2-4297-af96-111c8ab4838b admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:38:16.959003 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc3ce63-fde2-4297-af96-111c8ab4838b admin admin] PATH_INFO: `/v3/projects/213ccef9365044dc854b6da80f9c9b37` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:38:16.960256 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc3ce63-fde2-4297-af96-111c8ab4838b admin admin] RBAC: Authorizing `identity:delete_project(project_id=213ccef9365044dc854b6da80f9c9b37)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:38:16.962310 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc3ce63-fde2-4297-af96-111c8ab4838b admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:38:17.013690 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4167acbe-1966-4be7-afa4-ff743ec3b697 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:38:17.018458 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70319e4-502e-4406-81d0-56b71a6c3e7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:38:17.019084 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70319e4-502e-4406-81d0-56b71a6c3e7b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:38:17.019397 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70319e4-502e-4406-81d0-56b71a6c3e7b admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:38:17.019671 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70319e4-502e-4406-81d0-56b71a6c3e7b admin admin] PATH_INFO: `/v3/projects/8332c70b0109400795a3382bcf1b6705` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:38:17.020803 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70319e4-502e-4406-81d0-56b71a6c3e7b admin admin] RBAC: Authorizing `identity:delete_project(project_id=8332c70b0109400795a3382bcf1b6705)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:38:17.022711 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70319e4-502e-4406-81d0-56b71a6c3e7b admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:38:17.068767 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-cdc3ce63-fde2-4297-af96-111c8ab4838b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '213ccef9365044dc854b6da80f9c9b37', 'request_id': 'req-cdc3ce63-fde2-4297-af96-111c8ab4838b'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:38:17.074158 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-cdc3ce63-fde2-4297-af96-111c8ab4838b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '213ccef9365044dc854b6da80f9c9b37', 'request_id': 'req-cdc3ce63-fde2-4297-af96-111c8ab4838b'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:38:17.147676 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-d70319e4-502e-4406-81d0-56b71a6c3e7b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8332c70b0109400795a3382bcf1b6705', 'request_id': 'req-d70319e4-502e-4406-81d0-56b71a6c3e7b'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:38:17.159746 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-cdc3ce63-fde2-4297-af96-111c8ab4838b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '213ccef9365044dc854b6da80f9c9b37', 'request_id': 'req-cdc3ce63-fde2-4297-af96-111c8ab4838b'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:38:17.171166 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-cdc3ce63-fde2-4297-af96-111c8ab4838b admin admin] The token cache is being invalidate because project 213ccef9365044dc854b6da80f9c9b37 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81698) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 04:38:17.171458 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-cdc3ce63-fde2-4297-af96-111c8ab4838b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:38:17.172100 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-cdc3ce63-fde2-4297-af96-111c8ab4838b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:38:17.177008 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1145/2274] 10.222.0.30 () {58 vars in 1233 bytes} [Tue Sep 6 04:38:16 2022] DELETE /identity/v3/projects/213ccef9365044dc854b6da80f9c9b37 => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 04:38:17.178476 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2c66c3-604f-4524-a485-ff2c114165b3 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:38:17.178759 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2c66c3-604f-4524-a485-ff2c114165b3 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:38:17.179028 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2c66c3-604f-4524-a485-ff2c114165b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:38:17.207454 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-8e2c66c3-604f-4524-a485-ff2c114165b3 None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:38:17.208923 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-8e2c66c3-604f-4524-a485-ff2c114165b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:38:17.234007 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:DELETE=12,keystone:SELECT=87,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:38:17.256180 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-d70319e4-502e-4406-81d0-56b71a6c3e7b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8332c70b0109400795a3382bcf1b6705', 'request_id': 'req-d70319e4-502e-4406-81d0-56b71a6c3e7b'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:38:17.263831 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-d70319e4-502e-4406-81d0-56b71a6c3e7b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8332c70b0109400795a3382bcf1b6705', 'request_id': 'req-d70319e4-502e-4406-81d0-56b71a6c3e7b'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:38:17.276915 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1146/2275] 10.222.0.30 () {58 vars in 958 bytes} [Tue Sep 6 04:38:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 04:38:17.278852 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180ee01c-0331-4714-b9d5-ac3987411b5d None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:38:17.279088 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-d70319e4-502e-4406-81d0-56b71a6c3e7b admin admin] The token cache is being invalidate because project 8332c70b0109400795a3382bcf1b6705 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81697) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 04:38:17.279227 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180ee01c-0331-4714-b9d5-ac3987411b5d None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:38:17.279436 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-d70319e4-502e-4406-81d0-56b71a6c3e7b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:38:17.279541 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180ee01c-0331-4714-b9d5-ac3987411b5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:38:17.280493 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-d70319e4-502e-4406-81d0-56b71a6c3e7b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:38:17.282392 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1130/2276] 10.222.0.30 () {58 vars in 1233 bytes} [Tue Sep 6 04:38:16 2022] DELETE /identity/v3/projects/8332c70b0109400795a3382bcf1b6705 => generated 0 bytes in 340 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 04:38:17.288112 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b147e8c0-feb4-444b-bc5b-e9342d911fa1 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:38:17.288575 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b147e8c0-feb4-444b-bc5b-e9342d911fa1 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:38:17.289279 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b147e8c0-feb4-444b-bc5b-e9342d911fa1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:38:17.304559 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-180ee01c-0331-4714-b9d5-ac3987411b5d None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:38:17.305294 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-180ee01c-0331-4714-b9d5-ac3987411b5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:38:17.310821 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-b147e8c0-feb4-444b-bc5b-e9342d911fa1 None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:38:17.312106 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-b147e8c0-feb4-444b-bc5b-e9342d911fa1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:38:17.314008 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1147/2277] 10.222.0.30 () {58 vars in 958 bytes} [Tue Sep 6 04:38:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 04:38:17.318468 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180ee01c-0331-4714-b9d5-ac3987411b5d None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:38:17.325242 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1131/2278] 10.222.0.30 () {58 vars in 958 bytes} [Tue Sep 6 04:38:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 04:38:17.333583 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969bef1e-1021-4bb7-8a68-981bb68579eb None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:38:17.333945 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969bef1e-1021-4bb7-8a68-981bb68579eb None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:38:17.334289 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969bef1e-1021-4bb7-8a68-981bb68579eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:38:17.334562 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180ee01c-0331-4714-b9d5-ac3987411b5d None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:38:17.339367 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d2985d-6f72-4e1e-b12f-a818296f85d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:38:17.340000 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d2985d-6f72-4e1e-b12f-a818296f85d1 admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:38:17.340264 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d2985d-6f72-4e1e-b12f-a818296f85d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:38:17.340544 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d2985d-6f72-4e1e-b12f-a818296f85d1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:38:17.341203 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d2985d-6f72-4e1e-b12f-a818296f85d1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:38:17.341540 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d2985d-6f72-4e1e-b12f-a818296f85d1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:38:17.343149 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d2985d-6f72-4e1e-b12f-a818296f85d1 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:38:17.357394 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1148/2279] 10.222.0.30 () {58 vars in 1153 bytes} [Tue Sep 6 04:38:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:38:17.357589 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-969bef1e-1021-4bb7-8a68-981bb68579eb None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:38:17.358655 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-969bef1e-1021-4bb7-8a68-981bb68579eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:38:17.359727 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d2985d-6f72-4e1e-b12f-a818296f85d1 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:38:17.372230 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d2985d-6f72-4e1e-b12f-a818296f85d1 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:38:17.375072 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1132/2280] 10.222.0.30 () {58 vars in 958 bytes} [Tue Sep 6 04:38:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 04:38:17.377669 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29baa423-8635-4947-9b5e-47c1902b3907 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:38:17.378629 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29baa423-8635-4947-9b5e-47c1902b3907 admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:38:17.379043 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969bef1e-1021-4bb7-8a68-981bb68579eb None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:38:17.379372 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29baa423-8635-4947-9b5e-47c1902b3907 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:38:17.379635 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29baa423-8635-4947-9b5e-47c1902b3907 admin admin] PATH_INFO: `/v3/projects` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:38:17.380735 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29baa423-8635-4947-9b5e-47c1902b3907 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:38:17.382645 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29baa423-8635-4947-9b5e-47c1902b3907 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:38:17.388492 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969bef1e-1021-4bb7-8a68-981bb68579eb None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:38:17.393683 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484ae32e-e088-4d73-954c-f680cf0cfe07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:38:17.394296 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484ae32e-e088-4d73-954c-f680cf0cfe07 admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:38:17.394575 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484ae32e-e088-4d73-954c-f680cf0cfe07 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:38:17.394846 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484ae32e-e088-4d73-954c-f680cf0cfe07 admin admin] PATH_INFO: `/v3/domains` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:38:17.395490 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484ae32e-e088-4d73-954c-f680cf0cfe07 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:38:17.395788 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484ae32e-e088-4d73-954c-f680cf0cfe07 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:38:17.397387 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484ae32e-e088-4d73-954c-f680cf0cfe07 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:38:17.410680 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1133/2281] 10.222.0.30 () {58 vars in 1153 bytes} [Tue Sep 6 04:38:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 04:38:17.414844 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484ae32e-e088-4d73-954c-f680cf0cfe07 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:38:17.424781 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484ae32e-e088-4d73-954c-f680cf0cfe07 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:38:17.429417 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c419c389-0c4a-4dd1-890e-bb8331e16ae2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:38:17.430011 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c419c389-0c4a-4dd1-890e-bb8331e16ae2 admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:38:17.430277 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c419c389-0c4a-4dd1-890e-bb8331e16ae2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:38:17.430549 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c419c389-0c4a-4dd1-890e-bb8331e16ae2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:38:17.431166 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c419c389-0c4a-4dd1-890e-bb8331e16ae2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:38:17.433034 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c419c389-0c4a-4dd1-890e-bb8331e16ae2 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:38:17.447037 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1149/2282] 10.222.0.30 () {60 vars in 1153 bytes} [Tue Sep 6 04:38:17 2022] POST /identity/v3/projects => generated 383 bytes in 88 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 04:38:17.451309 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29baa423-8635-4947-9b5e-47c1902b3907 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:38:17.512991 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29baa423-8635-4947-9b5e-47c1902b3907 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:38:17.517837 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3574af20-f296-44ee-a3ca-923fd7d968d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:38:17.518445 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3574af20-f296-44ee-a3ca-923fd7d968d3 admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:38:17.518721 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3574af20-f296-44ee-a3ca-923fd7d968d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:38:17.518991 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3574af20-f296-44ee-a3ca-923fd7d968d3 admin admin] PATH_INFO: `/v3/users` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:38:17.519649 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3574af20-f296-44ee-a3ca-923fd7d968d3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:38:17.521588 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3574af20-f296-44ee-a3ca-923fd7d968d3 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:38:17.577775 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1134/2283] 10.222.0.30 () {60 vars in 1153 bytes} [Tue Sep 6 04:38:17 2022] POST /identity/v3/projects => generated 383 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 04:38:17.582345 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c419c389-0c4a-4dd1-890e-bb8331e16ae2 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:38:17.617299 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1150/2284] 10.222.0.30 () {60 vars in 1144 bytes} [Tue Sep 6 04:38:17 2022] POST /identity/v3/users => generated 292 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 04:38:17.621538 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3574af20-f296-44ee-a3ca-923fd7d968d3 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:38:17.632152 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3574af20-f296-44ee-a3ca-923fd7d968d3 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:38:17.655465 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c419c389-0c4a-4dd1-890e-bb8331e16ae2 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:38:17.660152 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15cd50f1-9482-4727-81f7-85f3ab9b41b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:38:17.660839 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cd50f1-9482-4727-81f7-85f3ab9b41b3 admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:38:17.661032 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cd50f1-9482-4727-81f7-85f3ab9b41b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:38:17.661299 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cd50f1-9482-4727-81f7-85f3ab9b41b3 admin admin] PATH_INFO: `/v3/users` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:38:17.661980 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15cd50f1-9482-4727-81f7-85f3ab9b41b3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:38:17.663911 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15cd50f1-9482-4727-81f7-85f3ab9b41b3 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:38:17.689708 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ce0f9c-d4d3-4e70-86c0-acf40e34e068 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:38:17.690329 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ce0f9c-d4d3-4e70-86c0-acf40e34e068 admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:38:17.690609 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ce0f9c-d4d3-4e70-86c0-acf40e34e068 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:38:17.690884 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ce0f9c-d4d3-4e70-86c0-acf40e34e068 admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:38:17.691566 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ce0f9c-d4d3-4e70-86c0-acf40e34e068 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:38:17.693201 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ce0f9c-d4d3-4e70-86c0-acf40e34e068 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:38:17.703135 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1151/2285] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Sep 6 04:38:17 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 04:38:17.707785 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ce0f9c-d4d3-4e70-86c0-acf40e34e068 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:38:17.718102 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ce0f9c-d4d3-4e70-86c0-acf40e34e068 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:38:17.723582 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28691133-9ad3-407f-8c74-d44fdebd07dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:38:17.724175 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28691133-9ad3-407f-8c74-d44fdebd07dc admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:38:17.724455 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28691133-9ad3-407f-8c74-d44fdebd07dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:38:17.724727 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28691133-9ad3-407f-8c74-d44fdebd07dc admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:38:17.725396 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28691133-9ad3-407f-8c74-d44fdebd07dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:38:17.726990 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28691133-9ad3-407f-8c74-d44fdebd07dc admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:38:17.737528 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1152/2286] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Sep 6 04:38:17 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 06 04:38:17.741513 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28691133-9ad3-407f-8c74-d44fdebd07dc admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:38:17.751020 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28691133-9ad3-407f-8c74-d44fdebd07dc admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:38:17.756526 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a78e73a-4a53-4d50-ae99-beb025bf6125 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:38:17.757221 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a78e73a-4a53-4d50-ae99-beb025bf6125 admin admin] REQUEST_METHOD: `PUT` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:38:17.757539 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a78e73a-4a53-4d50-ae99-beb025bf6125 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:38:17.757816 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a78e73a-4a53-4d50-ae99-beb025bf6125 admin admin] PATH_INFO: `/v3/projects/36495849f5124872ba74fe21b00f72fe/users/784815b8c6d0400c92caa54981b137d9/roles/29607b2de39f40d68bb6ceef64b8a732` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:38:17.774272 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1135/2287] 10.222.0.30 () {60 vars in 1144 bytes} [Tue Sep 6 04:38:17 2022] POST /identity/v3/users => generated 292 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 04:38:17.775370 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15cd50f1-9482-4727-81f7-85f3ab9b41b3 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:38:17.776523 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a78e73a-4a53-4d50-ae99-beb025bf6125 admin admin] RBAC: Authorizing `identity:create_grant(project_id=36495849f5124872ba74fe21b00f72fe, user_id=784815b8c6d0400c92caa54981b137d9, role_id=29607b2de39f40d68bb6ceef64b8a732)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:38:17.779232 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a78e73a-4a53-4d50-ae99-beb025bf6125 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:38:17.786472 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15cd50f1-9482-4727-81f7-85f3ab9b41b3 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:38:17.791151 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a158416-c97b-40bb-9f90-7a9bf544b1c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:38:17.791853 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a158416-c97b-40bb-9f90-7a9bf544b1c2 admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:38:17.792088 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a158416-c97b-40bb-9f90-7a9bf544b1c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:38:17.792395 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a158416-c97b-40bb-9f90-7a9bf544b1c2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:38:17.793038 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a158416-c97b-40bb-9f90-7a9bf544b1c2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:38:17.794678 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a158416-c97b-40bb-9f90-7a9bf544b1c2 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:38:17.804584 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1136/2288] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Sep 6 04:38:17 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 04:38:17.808425 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a158416-c97b-40bb-9f90-7a9bf544b1c2 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:38:17.818503 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a158416-c97b-40bb-9f90-7a9bf544b1c2 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:38:17.823129 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfcd1242-7d30-465e-8310-89a8ce6a1e57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:38:17.823712 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcd1242-7d30-465e-8310-89a8ce6a1e57 admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:38:17.823997 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcd1242-7d30-465e-8310-89a8ce6a1e57 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:38:17.824274 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcd1242-7d30-465e-8310-89a8ce6a1e57 admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:38:17.824896 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfcd1242-7d30-465e-8310-89a8ce6a1e57 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:38:17.826490 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfcd1242-7d30-465e-8310-89a8ce6a1e57 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:38:17.837371 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1137/2289] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Sep 6 04:38:17 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 06 04:38:17.841159 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfcd1242-7d30-465e-8310-89a8ce6a1e57 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:38:17.851148 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfcd1242-7d30-465e-8310-89a8ce6a1e57 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:38:17.860172 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a33fa11-6b58-4213-961f-e9ee49caf376 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:38:17.860172 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a33fa11-6b58-4213-961f-e9ee49caf376 admin admin] REQUEST_METHOD: `PUT` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:38:17.860172 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a33fa11-6b58-4213-961f-e9ee49caf376 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:38:17.860172 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a33fa11-6b58-4213-961f-e9ee49caf376 admin admin] PATH_INFO: `/v3/projects/83eff2d6b41d400899c6e6f1e7871f64/users/7a16cf3f57e34be99cb172044962b988/roles/29607b2de39f40d68bb6ceef64b8a732` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:38:17.873488 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1153/2290] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Sep 6 04:38:17 2022] PUT /identity/v3/projects/36495849f5124872ba74fe21b00f72fe/users/784815b8c6d0400c92caa54981b137d9/roles/29607b2de39f40d68bb6ceef64b8a732 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 04:38:17.873813 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a33fa11-6b58-4213-961f-e9ee49caf376 admin admin] RBAC: Authorizing `identity:create_grant(project_id=83eff2d6b41d400899c6e6f1e7871f64, user_id=7a16cf3f57e34be99cb172044962b988, role_id=29607b2de39f40d68bb6ceef64b8a732)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:38:17.877099 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a33fa11-6b58-4213-961f-e9ee49caf376 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:38:17.878954 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a78807a-5650-4a3a-828a-eb69b77010db None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:38:17.879425 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a78807a-5650-4a3a-828a-eb69b77010db None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:38:17.879626 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a78807a-5650-4a3a-828a-eb69b77010db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:38:17.899313 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-9a78807a-5650-4a3a-828a-eb69b77010db None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:38:17.900777 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-9a78807a-5650-4a3a-828a-eb69b77010db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:38:17.952940 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1138/2291] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Sep 6 04:38:17 2022] PUT /identity/v3/projects/83eff2d6b41d400899c6e6f1e7871f64/users/7a16cf3f57e34be99cb172044962b988/roles/29607b2de39f40d68bb6ceef64b8a732 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 04:38:17.963279 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150f5d7f-aae8-443a-9fc4-d2346aba6042 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:38:17.963566 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150f5d7f-aae8-443a-9fc4-d2346aba6042 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:38:17.963840 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150f5d7f-aae8-443a-9fc4-d2346aba6042 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:38:17.964870 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1154/2292] 10.222.0.30 () {58 vars in 958 bytes} [Tue Sep 6 04:38:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 04:38:17.974366 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a78807a-5650-4a3a-828a-eb69b77010db None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:38:17.975699 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-9a78807a-5650-4a3a-828a-eb69b77010db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:38:17.996757 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-150f5d7f-aae8-443a-9fc4-d2346aba6042 None None] MFA Rules not processed for user `7a16cf3f57e34be99cb172044962b988`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:38:17.998048 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-150f5d7f-aae8-443a-9fc4-d2346aba6042 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:38:18.036886 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a78807a-5650-4a3a-828a-eb69b77010db None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:38:18.066568 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1139/2293] 10.222.0.30 () {58 vars in 958 bytes} [Tue Sep 6 04:38:17 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 04:38:18.077825 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150f5d7f-aae8-443a-9fc4-d2346aba6042 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:38:18.079300 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-150f5d7f-aae8-443a-9fc4-d2346aba6042 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:38:18.096990 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21fae99b-5006-41a1-8331-2a9b8cb9b334 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90492440f6a6471e8392361c098428aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b707733ce65c42edbc1fd95ad0a1c06c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:38:18.098056 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fae99b-5006-41a1-8331-2a9b8cb9b334 service neutron] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:38:18.098482 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fae99b-5006-41a1-8331-2a9b8cb9b334 service neutron] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:38:18.098902 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fae99b-5006-41a1-8331-2a9b8cb9b334 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:38:18.124899 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:38:18.147568 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150f5d7f-aae8-443a-9fc4-d2346aba6042 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:38:18.151563 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e69738-d111-46dc-b5da-6bb4e2aeaaf1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd09b5ab675634a2eb4b03755743f7d0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b707733ce65c42edbc1fd95ad0a1c06c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:38:18.152221 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e69738-d111-46dc-b5da-6bb4e2aeaaf1 service cinder] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:38:18.152503 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e69738-d111-46dc-b5da-6bb4e2aeaaf1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:38:18.152774 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e69738-d111-46dc-b5da-6bb4e2aeaaf1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:38:18.161505 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e69738-d111-46dc-b5da-6bb4e2aeaaf1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:38:18.163720 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e69738-d111-46dc-b5da-6bb4e2aeaaf1 service cinder] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:38:18.166693 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21fae99b-5006-41a1-8331-2a9b8cb9b334 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:38:18.168069 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21fae99b-5006-41a1-8331-2a9b8cb9b334 service neutron] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:38:18.172873 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1140/2294] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Sep 6 04:38:18 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 06 04:38:18.175966 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e69738-d111-46dc-b5da-6bb4e2aeaaf1 service cinder] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:38:18.176081 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-b6e69738-d111-46dc-b5da-6bb4e2aeaaf1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:38:18.178075 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1155/2295] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Sep 6 04:38:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 204 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 06 04:38:18.188408 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e69738-d111-46dc-b5da-6bb4e2aeaaf1 service cinder] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:38:18.192849 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22de429e-77ca-4b65-a405-1a9fdb6bbfcb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd09b5ab675634a2eb4b03755743f7d0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b707733ce65c42edbc1fd95ad0a1c06c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:38:18.194117 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22de429e-77ca-4b65-a405-1a9fdb6bbfcb service cinder] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:38:18.194398 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22de429e-77ca-4b65-a405-1a9fdb6bbfcb service cinder] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:38:18.195123 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22de429e-77ca-4b65-a405-1a9fdb6bbfcb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:38:18.196508 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-22de429e-77ca-4b65-a405-1a9fdb6bbfcb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:38:18.257457 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22de429e-77ca-4b65-a405-1a9fdb6bbfcb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:38:18.258897 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22de429e-77ca-4b65-a405-1a9fdb6bbfcb service cinder] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:38:18.271099 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1141/2296] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Sep 6 04:38:18 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 04:38:18.290819 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:38:19.056676 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:38:19.449177 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cb500c-18b4-4d7b-b2bd-aaa72c6a635a None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:38:19.449620 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cb500c-18b4-4d7b-b2bd-aaa72c6a635a None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:38:19.449953 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cb500c-18b4-4d7b-b2bd-aaa72c6a635a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:38:19.482823 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-18cb500c-18b4-4d7b-b2bd-aaa72c6a635a None None] MFA Rules not processed for user `784815b8c6d0400c92caa54981b137d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:38:19.484036 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-18cb500c-18b4-4d7b-b2bd-aaa72c6a635a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:38:19.572679 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1156/2297] 10.222.0.30 () {58 vars in 958 bytes} [Tue Sep 6 04:38:19 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 04:38:19.583157 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22de429e-77ca-4b65-a405-1a9fdb6bbfcb service cinder] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:38:19.591650 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22de429e-77ca-4b65-a405-1a9fdb6bbfcb service cinder] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:38:19.595615 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee08249-1fd1-47c0-9cbb-c0325b613375 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd09b5ab675634a2eb4b03755743f7d0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b707733ce65c42edbc1fd95ad0a1c06c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:38:19.596200 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee08249-1fd1-47c0-9cbb-c0325b613375 service cinder] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:38:19.596484 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee08249-1fd1-47c0-9cbb-c0325b613375 service cinder] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:38:19.596752 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee08249-1fd1-47c0-9cbb-c0325b613375 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:38:19.605141 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee08249-1fd1-47c0-9cbb-c0325b613375 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:38:19.606670 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee08249-1fd1-47c0-9cbb-c0325b613375 service cinder] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:38:19.616685 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1142/2298] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Sep 6 04:38:19 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 04:38:20.124571 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18cb500c-18b4-4d7b-b2bd-aaa72c6a635a None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:38:20.134759 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18cb500c-18b4-4d7b-b2bd-aaa72c6a635a None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:38:20.138321 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60409dc3-4c71-4e23-806e-3cc971709e35 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd09b5ab675634a2eb4b03755743f7d0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b707733ce65c42edbc1fd95ad0a1c06c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:38:20.138918 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60409dc3-4c71-4e23-806e-3cc971709e35 service cinder] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:38:20.139191 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60409dc3-4c71-4e23-806e-3cc971709e35 service cinder] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:38:20.139499 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60409dc3-4c71-4e23-806e-3cc971709e35 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:38:20.141202 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-60409dc3-4c71-4e23-806e-3cc971709e35 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:38:20.154836 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:38:20.218527 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60409dc3-4c71-4e23-806e-3cc971709e35 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:38:20.218527 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60409dc3-4c71-4e23-806e-3cc971709e35 service cinder] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:38:20.228916 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1157/2299] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Sep 6 04:38:20 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 04:38:22.395004 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee08249-1fd1-47c0-9cbb-c0325b613375 service cinder] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:38:22.404062 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acb9607-d807-478d-a6b6-ee0c118eb994 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:38:22.404494 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acb9607-d807-478d-a6b6-ee0c118eb994 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:38:22.404609 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acb9607-d807-478d-a6b6-ee0c118eb994 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:38:22.417170 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee08249-1fd1-47c0-9cbb-c0325b613375 service cinder] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:38:22.422390 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc47096f-0542-4be6-8a85-c3bac1c380a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:38:22.423145 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc47096f-0542-4be6-8a85-c3bac1c380a5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:38:22.423439 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc47096f-0542-4be6-8a85-c3bac1c380a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:38:22.423717 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc47096f-0542-4be6-8a85-c3bac1c380a5 admin admin] PATH_INFO: `/v3/users/7a16cf3f57e34be99cb172044962b988` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:38:22.424988 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-1acb9607-d807-478d-a6b6-ee0c118eb994 None None] MFA Rules not processed for user `784815b8c6d0400c92caa54981b137d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:38:22.425192 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc47096f-0542-4be6-8a85-c3bac1c380a5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7a16cf3f57e34be99cb172044962b988)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:38:22.426342 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-1acb9607-d807-478d-a6b6-ee0c118eb994 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:38:22.427176 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc47096f-0542-4be6-8a85-c3bac1c380a5 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:38:22.432633 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:38:22.735843 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1158/2300] 10.222.0.30 () {58 vars in 958 bytes} [Tue Sep 6 04:38:22 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 04:38:23.030527 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-dc47096f-0542-4be6-8a85-c3bac1c380a5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7a16cf3f57e34be99cb172044962b988'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:38:23.119342 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-dc47096f-0542-4be6-8a85-c3bac1c380a5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7a16cf3f57e34be99cb172044962b988'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:38:23.120428 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-dc47096f-0542-4be6-8a85-c3bac1c380a5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7a16cf3f57e34be99cb172044962b988'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:38:23.151197 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-dc47096f-0542-4be6-8a85-c3bac1c380a5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7a16cf3f57e34be99cb172044962b988'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:38:23.170986 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-dc47096f-0542-4be6-8a85-c3bac1c380a5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7a16cf3f57e34be99cb172044962b988'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:38:23.182504 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1143/2301] 10.222.0.30 () {58 vars in 1224 bytes} [Tue Sep 6 04:38:22 2022] DELETE /identity/v3/users/7a16cf3f57e34be99cb172044962b988 => generated 0 bytes in 788 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 04:38:23.186755 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a598c6-2f3c-4882-9f90-f591df35e9c4 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:38:23.187157 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a598c6-2f3c-4882-9f90-f591df35e9c4 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:38:23.187666 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a598c6-2f3c-4882-9f90-f591df35e9c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:38:23.211568 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-91a598c6-2f3c-4882-9f90-f591df35e9c4 None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:38:23.212798 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-91a598c6-2f3c-4882-9f90-f591df35e9c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:38:23.224659 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b56dd6-7adc-407f-919b-d90c95275eea None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:38:23.224953 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b56dd6-7adc-407f-919b-d90c95275eea None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:38:23.225238 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b56dd6-7adc-407f-919b-d90c95275eea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:38:23.245811 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-f1b56dd6-7adc-407f-919b-d90c95275eea None None] MFA Rules not processed for user `4b6c761af9c5457ea1e481ce400aded8`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:38:23.247190 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-f1b56dd6-7adc-407f-919b-d90c95275eea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:38:23.320440 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1144/2302] 10.222.0.30 () {58 vars in 958 bytes} [Tue Sep 6 04:38:23 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 04:38:23.330269 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1159/2303] 10.222.0.30 () {58 vars in 958 bytes} [Tue Sep 6 04:38:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 04:38:23.340256 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b56dd6-7adc-407f-919b-d90c95275eea None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:38:23.342249 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-f1b56dd6-7adc-407f-919b-d90c95275eea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:38:23.415727 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b56dd6-7adc-407f-919b-d90c95275eea None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:38:23.418055 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67b52b1-c92e-48a9-91fa-927761b2e016 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90492440f6a6471e8392361c098428aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b707733ce65c42edbc1fd95ad0a1c06c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:38:23.418752 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67b52b1-c92e-48a9-91fa-927761b2e016 service neutron] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:38:23.419087 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67b52b1-c92e-48a9-91fa-927761b2e016 service neutron] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:38:23.419367 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67b52b1-c92e-48a9-91fa-927761b2e016 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:38:23.430203 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67b52b1-c92e-48a9-91fa-927761b2e016 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:38:23.431819 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67b52b1-c92e-48a9-91fa-927761b2e016 service neutron] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:38:23.442766 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1145/2304] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Sep 6 04:38:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 04:38:23.920839 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a598c6-2f3c-4882-9f90-f591df35e9c4 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:38:23.922490 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-91a598c6-2f3c-4882-9f90-f591df35e9c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:38:23.943570 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a598c6-2f3c-4882-9f90-f591df35e9c4 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:38:23.960332 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8521cf-bc1b-4560-bb6b-13ff07a17d97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:38:23.963318 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8521cf-bc1b-4560-bb6b-13ff07a17d97 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:38:23.963318 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8521cf-bc1b-4560-bb6b-13ff07a17d97 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:38:23.963318 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8521cf-bc1b-4560-bb6b-13ff07a17d97 admin admin] PATH_INFO: `/v3/projects/83eff2d6b41d400899c6e6f1e7871f64` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:38:23.963318 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a8521cf-bc1b-4560-bb6b-13ff07a17d97 admin admin] RBAC: Authorizing `identity:delete_project(project_id=83eff2d6b41d400899c6e6f1e7871f64)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:38:23.964853 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a8521cf-bc1b-4560-bb6b-13ff07a17d97 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:38:24.057722 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-7a8521cf-bc1b-4560-bb6b-13ff07a17d97 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '83eff2d6b41d400899c6e6f1e7871f64', 'request_id': 'req-7a8521cf-bc1b-4560-bb6b-13ff07a17d97'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:38:24.064033 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-7a8521cf-bc1b-4560-bb6b-13ff07a17d97 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '83eff2d6b41d400899c6e6f1e7871f64', 'request_id': 'req-7a8521cf-bc1b-4560-bb6b-13ff07a17d97'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:38:24.118911 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-7a8521cf-bc1b-4560-bb6b-13ff07a17d97 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '83eff2d6b41d400899c6e6f1e7871f64', 'request_id': 'req-7a8521cf-bc1b-4560-bb6b-13ff07a17d97'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:38:24.131359 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-7a8521cf-bc1b-4560-bb6b-13ff07a17d97 admin admin] The token cache is being invalidate because project 83eff2d6b41d400899c6e6f1e7871f64 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81698) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 04:38:24.131994 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-7a8521cf-bc1b-4560-bb6b-13ff07a17d97 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:38:24.132646 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-7a8521cf-bc1b-4560-bb6b-13ff07a17d97 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:38:24.134251 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1160/2305] 10.222.0.30 () {58 vars in 1233 bytes} [Tue Sep 6 04:38:23 2022] DELETE /identity/v3/projects/83eff2d6b41d400899c6e6f1e7871f64 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 04:38:24.139702 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef81ea24-89f6-4054-af94-f10b800e03c8 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:38:24.139883 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef81ea24-89f6-4054-af94-f10b800e03c8 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:38:24.140184 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef81ea24-89f6-4054-af94-f10b800e03c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:38:24.160950 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-ef81ea24-89f6-4054-af94-f10b800e03c8 None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:38:24.162210 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-ef81ea24-89f6-4054-af94-f10b800e03c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:38:24.193728 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:38:24.227780 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1146/2306] 10.222.0.30 () {58 vars in 958 bytes} [Tue Sep 6 04:38:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 04:38:24.237582 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4258ad-c676-4c9a-8b26-a9300c50caec None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:38:24.238560 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4258ad-c676-4c9a-8b26-a9300c50caec None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:38:24.239031 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4258ad-c676-4c9a-8b26-a9300c50caec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:38:24.276026 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-ac4258ad-c676-4c9a-8b26-a9300c50caec None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:38:24.277688 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-ac4258ad-c676-4c9a-8b26-a9300c50caec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:38:24.290794 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1161/2307] 10.222.0.30 () {58 vars in 958 bytes} [Tue Sep 6 04:38:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 04:38:24.292307 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef81ea24-89f6-4054-af94-f10b800e03c8 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:38:24.303645 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef81ea24-89f6-4054-af94-f10b800e03c8 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:38:24.309468 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041e3478-aef3-4365-aeb6-a8ccb8455342 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:38:24.310044 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041e3478-aef3-4365-aeb6-a8ccb8455342 admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:38:24.310328 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041e3478-aef3-4365-aeb6-a8ccb8455342 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:38:24.310662 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041e3478-aef3-4365-aeb6-a8ccb8455342 admin admin] PATH_INFO: `/v3/domains` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:38:24.311260 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-041e3478-aef3-4365-aeb6-a8ccb8455342 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:38:24.311563 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-041e3478-aef3-4365-aeb6-a8ccb8455342 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:38:24.313849 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-041e3478-aef3-4365-aeb6-a8ccb8455342 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:38:24.334008 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1147/2308] 10.222.0.30 () {58 vars in 1153 bytes} [Tue Sep 6 04:38:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:38:24.335130 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4258ad-c676-4c9a-8b26-a9300c50caec None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:38:24.345261 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4258ad-c676-4c9a-8b26-a9300c50caec None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:38:24.350524 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7dd3c6f-17c4-4d87-8c0e-e3d46f7c26d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:38:24.351222 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7dd3c6f-17c4-4d87-8c0e-e3d46f7c26d7 admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:38:24.351572 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7dd3c6f-17c4-4d87-8c0e-e3d46f7c26d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:38:24.351943 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7dd3c6f-17c4-4d87-8c0e-e3d46f7c26d7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:38:24.352735 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7dd3c6f-17c4-4d87-8c0e-e3d46f7c26d7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:38:24.354754 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7dd3c6f-17c4-4d87-8c0e-e3d46f7c26d7 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:38:24.428505 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:38:24.442703 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1162/2309] 10.222.0.30 () {60 vars in 1153 bytes} [Tue Sep 6 04:38:24 2022] POST /identity/v3/projects => generated 371 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 04:38:24.446501 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041e3478-aef3-4365-aeb6-a8ccb8455342 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:38:24.457898 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041e3478-aef3-4365-aeb6-a8ccb8455342 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:38:24.546419 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06aacadd-f3d1-4ec9-b198-fdc0243d7917 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:38:24.547136 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06aacadd-f3d1-4ec9-b198-fdc0243d7917 admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:38:24.547488 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06aacadd-f3d1-4ec9-b198-fdc0243d7917 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:38:24.547844 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06aacadd-f3d1-4ec9-b198-fdc0243d7917 admin admin] PATH_INFO: `/v3/users` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:38:24.549032 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06aacadd-f3d1-4ec9-b198-fdc0243d7917 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:38:24.552149 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06aacadd-f3d1-4ec9-b198-fdc0243d7917 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:38:24.663916 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1148/2310] 10.222.0.30 () {60 vars in 1144 bytes} [Tue Sep 6 04:38:24 2022] POST /identity/v3/users => generated 286 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 04:38:24.668711 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7dd3c6f-17c4-4d87-8c0e-e3d46f7c26d7 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:38:24.679338 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7dd3c6f-17c4-4d87-8c0e-e3d46f7c26d7 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:38:24.697101 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ce17af-b4d2-4400-8540-08c3bd05aa1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:38:24.697850 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ce17af-b4d2-4400-8540-08c3bd05aa1e admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:38:24.698201 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ce17af-b4d2-4400-8540-08c3bd05aa1e admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:38:24.698564 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ce17af-b4d2-4400-8540-08c3bd05aa1e admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:38:24.699269 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ce17af-b4d2-4400-8540-08c3bd05aa1e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:38:24.714046 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ce17af-b4d2-4400-8540-08c3bd05aa1e admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:38:24.731008 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1163/2311] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Sep 6 04:38:24 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 06 04:38:24.732124 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06aacadd-f3d1-4ec9-b198-fdc0243d7917 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:38:24.745485 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06aacadd-f3d1-4ec9-b198-fdc0243d7917 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:38:24.754387 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad84379-10e8-45d2-b664-fda33617dc79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:38:24.755346 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad84379-10e8-45d2-b664-fda33617dc79 admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:38:24.756352 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad84379-10e8-45d2-b664-fda33617dc79 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:38:24.756795 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad84379-10e8-45d2-b664-fda33617dc79 admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:38:24.757718 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad84379-10e8-45d2-b664-fda33617dc79 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:38:24.760738 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad84379-10e8-45d2-b664-fda33617dc79 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:38:24.775047 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1149/2312] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Sep 6 04:38:24 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 06 04:38:24.779538 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ce17af-b4d2-4400-8540-08c3bd05aa1e admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:38:24.792749 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ce17af-b4d2-4400-8540-08c3bd05aa1e admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:38:24.796752 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f6aded-aad4-45b4-92b0-3019404589ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:38:24.797522 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f6aded-aad4-45b4-92b0-3019404589ae admin admin] REQUEST_METHOD: `PUT` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:38:24.797838 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f6aded-aad4-45b4-92b0-3019404589ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:38:24.798119 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f6aded-aad4-45b4-92b0-3019404589ae admin admin] PATH_INFO: `/v3/projects/17ca991debba43b98c1ad8f0f91c63a9/users/65a8d2df1eed492a874ddaae3235edd0/roles/29607b2de39f40d68bb6ceef64b8a732` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:38:24.828461 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f6aded-aad4-45b4-92b0-3019404589ae admin admin] RBAC: Authorizing `identity:create_grant(project_id=17ca991debba43b98c1ad8f0f91c63a9, user_id=65a8d2df1eed492a874ddaae3235edd0, role_id=29607b2de39f40d68bb6ceef64b8a732)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:38:24.834209 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f6aded-aad4-45b4-92b0-3019404589ae admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:38:24.921033 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1164/2313] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Sep 6 04:38:24 2022] PUT /identity/v3/projects/17ca991debba43b98c1ad8f0f91c63a9/users/65a8d2df1eed492a874ddaae3235edd0/roles/29607b2de39f40d68bb6ceef64b8a732 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 04:38:24.926501 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81efc75f-0816-4f52-aa12-73f7b3577025 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:38:24.926794 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81efc75f-0816-4f52-aa12-73f7b3577025 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:38:24.927066 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81efc75f-0816-4f52-aa12-73f7b3577025 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:38:24.952487 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-81efc75f-0816-4f52-aa12-73f7b3577025 None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:38:24.954079 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-81efc75f-0816-4f52-aa12-73f7b3577025 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:38:25.034042 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1150/2314] 10.222.0.30 () {58 vars in 958 bytes} [Tue Sep 6 04:38:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 04:38:25.043198 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f6aded-aad4-45b4-92b0-3019404589ae admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:38:25.045802 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-f8f6aded-aad4-45b4-92b0-3019404589ae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:38:25.133964 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f6aded-aad4-45b4-92b0-3019404589ae admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:38:25.139154 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc5fda1-f8ed-4bf5-8d93-30987494bb05 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90492440f6a6471e8392361c098428aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b707733ce65c42edbc1fd95ad0a1c06c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:38:25.140772 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc5fda1-f8ed-4bf5-8d93-30987494bb05 service neutron] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:38:25.141419 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc5fda1-f8ed-4bf5-8d93-30987494bb05 service neutron] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:38:25.141978 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc5fda1-f8ed-4bf5-8d93-30987494bb05 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:38:25.156067 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc5fda1-f8ed-4bf5-8d93-30987494bb05 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:38:25.157619 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc5fda1-f8ed-4bf5-8d93-30987494bb05 service neutron] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:38:25.180967 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1165/2315] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Sep 6 04:38:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 04:38:27.707866 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81efc75f-0816-4f52-aa12-73f7b3577025 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:38:27.709782 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-81efc75f-0816-4f52-aa12-73f7b3577025 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:38:27.724007 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81efc75f-0816-4f52-aa12-73f7b3577025 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:38:27.729877 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ef9acf-81b7-461f-841f-30257873111d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:38:27.729877 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ef9acf-81b7-461f-841f-30257873111d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:38:27.730214 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ef9acf-81b7-461f-841f-30257873111d admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:38:27.730439 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ef9acf-81b7-461f-841f-30257873111d admin admin] PATH_INFO: `/v3/users/8eb559b8c3d34d3e86b7d52dffd4dccb` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:38:27.731989 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ef9acf-81b7-461f-841f-30257873111d admin admin] RBAC: Authorizing `identity:delete_user(user_id=8eb559b8c3d34d3e86b7d52dffd4dccb)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:38:27.734733 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ef9acf-81b7-461f-841f-30257873111d admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:38:27.978212 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:38:27.993056 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-e8ef9acf-81b7-461f-841f-30257873111d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8eb559b8c3d34d3e86b7d52dffd4dccb'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:38:28.087073 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-e8ef9acf-81b7-461f-841f-30257873111d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8eb559b8c3d34d3e86b7d52dffd4dccb'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:38:28.089185 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-e8ef9acf-81b7-461f-841f-30257873111d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8eb559b8c3d34d3e86b7d52dffd4dccb'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:38:28.103943 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-e8ef9acf-81b7-461f-841f-30257873111d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8eb559b8c3d34d3e86b7d52dffd4dccb'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:38:28.138662 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-e8ef9acf-81b7-461f-841f-30257873111d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8eb559b8c3d34d3e86b7d52dffd4dccb'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:38:28.152622 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1151/2316] 10.222.0.30 () {58 vars in 1224 bytes} [Tue Sep 6 04:38:27 2022] DELETE /identity/v3/users/8eb559b8c3d34d3e86b7d52dffd4dccb => generated 0 bytes in 446 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 04:38:28.157193 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68797926-218c-4735-ad37-be6bf010a1b3 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:38:28.159219 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68797926-218c-4735-ad37-be6bf010a1b3 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:38:28.159219 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68797926-218c-4735-ad37-be6bf010a1b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:38:28.185201 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-68797926-218c-4735-ad37-be6bf010a1b3 None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:38:28.187065 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-68797926-218c-4735-ad37-be6bf010a1b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:38:28.300010 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1166/2317] 10.222.0.30 () {58 vars in 958 bytes} [Tue Sep 6 04:38:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 04:38:28.317545 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ef9acf-81b7-461f-841f-30257873111d admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:38:28.319670 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-e8ef9acf-81b7-461f-841f-30257873111d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:38:28.411740 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ef9acf-81b7-461f-841f-30257873111d admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:38:28.415691 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d4e4fe-fbf8-42b5-822a-0dc102a0db0f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90492440f6a6471e8392361c098428aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b707733ce65c42edbc1fd95ad0a1c06c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:38:28.416313 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d4e4fe-fbf8-42b5-822a-0dc102a0db0f service neutron] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:38:28.416592 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d4e4fe-fbf8-42b5-822a-0dc102a0db0f service neutron] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:38:28.416863 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d4e4fe-fbf8-42b5-822a-0dc102a0db0f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:38:28.428909 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d4e4fe-fbf8-42b5-822a-0dc102a0db0f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:38:28.431127 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d4e4fe-fbf8-42b5-822a-0dc102a0db0f service neutron] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:38:28.448938 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1152/2318] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Sep 6 04:38:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 04:38:29.811122 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68797926-218c-4735-ad37-be6bf010a1b3 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:38:29.815642 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-68797926-218c-4735-ad37-be6bf010a1b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:38:29.832920 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68797926-218c-4735-ad37-be6bf010a1b3 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:38:29.838596 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3409227-819a-4afb-9a42-44f400682b24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:38:29.846959 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3409227-819a-4afb-9a42-44f400682b24 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:38:29.846959 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3409227-819a-4afb-9a42-44f400682b24 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:38:29.846959 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3409227-819a-4afb-9a42-44f400682b24 admin admin] PATH_INFO: `/v3/projects/7e953380840c42f7b7904aa0dc5638fc` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:38:29.846959 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3409227-819a-4afb-9a42-44f400682b24 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7e953380840c42f7b7904aa0dc5638fc)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:38:29.846959 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3409227-819a-4afb-9a42-44f400682b24 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:38:29.988600 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-a3409227-819a-4afb-9a42-44f400682b24 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7e953380840c42f7b7904aa0dc5638fc', 'request_id': 'req-a3409227-819a-4afb-9a42-44f400682b24'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:38:29.994689 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-a3409227-819a-4afb-9a42-44f400682b24 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7e953380840c42f7b7904aa0dc5638fc', 'request_id': 'req-a3409227-819a-4afb-9a42-44f400682b24'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:38:29.997204 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=92,keystone:INSERT=2,keystone:DELETE=6 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:38:30.078803 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-a3409227-819a-4afb-9a42-44f400682b24 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7e953380840c42f7b7904aa0dc5638fc', 'request_id': 'req-a3409227-819a-4afb-9a42-44f400682b24'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:38:30.090810 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-a3409227-819a-4afb-9a42-44f400682b24 admin admin] The token cache is being invalidate because project 7e953380840c42f7b7904aa0dc5638fc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81698) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 04:38:30.091101 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-a3409227-819a-4afb-9a42-44f400682b24 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:38:30.091752 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-a3409227-819a-4afb-9a42-44f400682b24 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:38:30.093242 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1167/2319] 10.222.0.30 () {58 vars in 1233 bytes} [Tue Sep 6 04:38:29 2022] DELETE /identity/v3/projects/7e953380840c42f7b7904aa0dc5638fc => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 04:38:30.097523 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d4e4fe-fbf8-42b5-822a-0dc102a0db0f service neutron] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:38:30.109247 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d4e4fe-fbf8-42b5-822a-0dc102a0db0f service neutron] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:38:30.179080 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58c4ac6-50ee-4853-865a-7fc249d2bc3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:38:30.179730 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58c4ac6-50ee-4853-865a-7fc249d2bc3c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:38:30.180115 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58c4ac6-50ee-4853-865a-7fc249d2bc3c admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:38:30.180696 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58c4ac6-50ee-4853-865a-7fc249d2bc3c admin admin] PATH_INFO: `/v3/users/bdc674a9b6a04d84abbbcba4c16cc4b1` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:38:30.181912 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58c4ac6-50ee-4853-865a-7fc249d2bc3c admin admin] RBAC: Authorizing `identity:delete_user(user_id=bdc674a9b6a04d84abbbcba4c16cc4b1)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:38:30.184789 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58c4ac6-50ee-4853-865a-7fc249d2bc3c admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:38:30.461896 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-a58c4ac6-50ee-4853-865a-7fc249d2bc3c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bdc674a9b6a04d84abbbcba4c16cc4b1'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:38:30.553493 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-a58c4ac6-50ee-4853-865a-7fc249d2bc3c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bdc674a9b6a04d84abbbcba4c16cc4b1'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:38:30.554291 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-a58c4ac6-50ee-4853-865a-7fc249d2bc3c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bdc674a9b6a04d84abbbcba4c16cc4b1'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:38:30.557050 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:38:30.582821 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-a58c4ac6-50ee-4853-865a-7fc249d2bc3c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bdc674a9b6a04d84abbbcba4c16cc4b1'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:38:30.618006 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-a58c4ac6-50ee-4853-865a-7fc249d2bc3c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bdc674a9b6a04d84abbbcba4c16cc4b1'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:38:30.630845 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1153/2320] 10.222.0.30 () {58 vars in 1224 bytes} [Tue Sep 6 04:38:30 2022] DELETE /identity/v3/users/bdc674a9b6a04d84abbbcba4c16cc4b1 => generated 0 bytes in 529 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 04:38:31.208603 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3409227-819a-4afb-9a42-44f400682b24 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:38:31.211474 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-a3409227-819a-4afb-9a42-44f400682b24 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:38:31.241620 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3409227-819a-4afb-9a42-44f400682b24 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:38:31.327362 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b886f53f-75eb-4cc7-a8e5-788de18acded admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:38:31.328010 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b886f53f-75eb-4cc7-a8e5-788de18acded admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:38:31.328450 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b886f53f-75eb-4cc7-a8e5-788de18acded admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:38:31.328791 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b886f53f-75eb-4cc7-a8e5-788de18acded admin admin] PATH_INFO: `/v3/projects/b7bb7cded1a64fc4a3128e7abc625472` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:38:31.329874 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b886f53f-75eb-4cc7-a8e5-788de18acded admin admin] RBAC: Authorizing `identity:delete_project(project_id=b7bb7cded1a64fc4a3128e7abc625472)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:38:31.332786 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b886f53f-75eb-4cc7-a8e5-788de18acded admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:38:31.440645 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-b886f53f-75eb-4cc7-a8e5-788de18acded admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b7bb7cded1a64fc4a3128e7abc625472', 'request_id': 'req-b886f53f-75eb-4cc7-a8e5-788de18acded'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:38:31.447716 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-b886f53f-75eb-4cc7-a8e5-788de18acded admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b7bb7cded1a64fc4a3128e7abc625472', 'request_id': 'req-b886f53f-75eb-4cc7-a8e5-788de18acded'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:38:31.490446 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-b886f53f-75eb-4cc7-a8e5-788de18acded admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b7bb7cded1a64fc4a3128e7abc625472', 'request_id': 'req-b886f53f-75eb-4cc7-a8e5-788de18acded'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:38:31.505330 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-b886f53f-75eb-4cc7-a8e5-788de18acded admin admin] The token cache is being invalidate because project b7bb7cded1a64fc4a3128e7abc625472 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81698) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 04:38:31.505754 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-b886f53f-75eb-4cc7-a8e5-788de18acded admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:38:31.506863 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-b886f53f-75eb-4cc7-a8e5-788de18acded admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:38:31.508639 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1168/2321] 10.222.0.30 () {58 vars in 1233 bytes} [Tue Sep 6 04:38:31 2022] DELETE /identity/v3/projects/b7bb7cded1a64fc4a3128e7abc625472 => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 04:38:31.512563 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58c4ac6-50ee-4853-865a-7fc249d2bc3c admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:38:31.514280 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-a58c4ac6-50ee-4853-865a-7fc249d2bc3c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:38:31.532617 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58c4ac6-50ee-4853-865a-7fc249d2bc3c admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:38:31.606107 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61487e7f-c915-474b-ab97-d90ebcbc3699 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:38:31.606662 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61487e7f-c915-474b-ab97-d90ebcbc3699 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:38:31.606943 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61487e7f-c915-474b-ab97-d90ebcbc3699 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:38:31.607217 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61487e7f-c915-474b-ab97-d90ebcbc3699 admin admin] PATH_INFO: `/v3/users/1462b2564e0d414591dc6f137e391d1d` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:38:31.609387 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61487e7f-c915-474b-ab97-d90ebcbc3699 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1462b2564e0d414591dc6f137e391d1d)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:38:31.612676 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61487e7f-c915-474b-ab97-d90ebcbc3699 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:38:31.815161 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-61487e7f-c915-474b-ab97-d90ebcbc3699 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1462b2564e0d414591dc6f137e391d1d'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:38:31.953453 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-61487e7f-c915-474b-ab97-d90ebcbc3699 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1462b2564e0d414591dc6f137e391d1d'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:38:31.954371 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-61487e7f-c915-474b-ab97-d90ebcbc3699 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1462b2564e0d414591dc6f137e391d1d'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:38:31.963895 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-61487e7f-c915-474b-ab97-d90ebcbc3699 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1462b2564e0d414591dc6f137e391d1d'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:38:31.977654 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-61487e7f-c915-474b-ab97-d90ebcbc3699 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1462b2564e0d414591dc6f137e391d1d'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:38:31.987272 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1154/2322] 10.222.0.30 () {58 vars in 1224 bytes} [Tue Sep 6 04:38:31 2022] DELETE /identity/v3/users/1462b2564e0d414591dc6f137e391d1d => generated 0 bytes in 474 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 04:38:32.545236 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b886f53f-75eb-4cc7-a8e5-788de18acded admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:38:32.555299 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-b886f53f-75eb-4cc7-a8e5-788de18acded admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:38:32.572597 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b886f53f-75eb-4cc7-a8e5-788de18acded admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:38:32.661688 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e4d061-b6fc-4fc4-9269-2c0d174eb046 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:38:32.662457 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e4d061-b6fc-4fc4-9269-2c0d174eb046 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:38:32.662816 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e4d061-b6fc-4fc4-9269-2c0d174eb046 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:38:32.663890 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e4d061-b6fc-4fc4-9269-2c0d174eb046 admin admin] PATH_INFO: `/v3/projects/b9d0c78786d1440d9fcbf8c174fe846b` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:38:32.668137 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e4d061-b6fc-4fc4-9269-2c0d174eb046 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b9d0c78786d1440d9fcbf8c174fe846b)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:38:32.670179 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e4d061-b6fc-4fc4-9269-2c0d174eb046 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:38:32.699270 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:DELETE=8,keystone:SELECT=91,keystone:INSERT=1 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:38:32.806328 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-54e4d061-b6fc-4fc4-9269-2c0d174eb046 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b9d0c78786d1440d9fcbf8c174fe846b', 'request_id': 'req-54e4d061-b6fc-4fc4-9269-2c0d174eb046'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:38:32.817144 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-54e4d061-b6fc-4fc4-9269-2c0d174eb046 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b9d0c78786d1440d9fcbf8c174fe846b', 'request_id': 'req-54e4d061-b6fc-4fc4-9269-2c0d174eb046'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:38:32.872961 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-54e4d061-b6fc-4fc4-9269-2c0d174eb046 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b9d0c78786d1440d9fcbf8c174fe846b', 'request_id': 'req-54e4d061-b6fc-4fc4-9269-2c0d174eb046'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:38:32.887620 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-54e4d061-b6fc-4fc4-9269-2c0d174eb046 admin admin] The token cache is being invalidate because project b9d0c78786d1440d9fcbf8c174fe846b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81698) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 04:38:32.887970 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-54e4d061-b6fc-4fc4-9269-2c0d174eb046 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:38:32.888743 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-54e4d061-b6fc-4fc4-9269-2c0d174eb046 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:38:32.894140 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1169/2323] 10.222.0.30 () {58 vars in 1233 bytes} [Tue Sep 6 04:38:32 2022] DELETE /identity/v3/projects/b9d0c78786d1440d9fcbf8c174fe846b => generated 0 bytes in 346 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 04:38:32.896066 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd47d1da-69d7-4995-8f9f-8df9db343786 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:38:32.896551 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd47d1da-69d7-4995-8f9f-8df9db343786 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:38:32.896660 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd47d1da-69d7-4995-8f9f-8df9db343786 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:38:32.919425 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-dd47d1da-69d7-4995-8f9f-8df9db343786 None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:38:32.920936 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-dd47d1da-69d7-4995-8f9f-8df9db343786 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:38:32.980326 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:38:33.002919 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1155/2324] 10.222.0.30 () {58 vars in 958 bytes} [Tue Sep 6 04:38:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 04:38:33.004584 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bd38f8-2c20-4b77-8b91-0d6db56348cb None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:38:33.004863 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bd38f8-2c20-4b77-8b91-0d6db56348cb None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:38:33.005141 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bd38f8-2c20-4b77-8b91-0d6db56348cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:38:33.028736 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-f9bd38f8-2c20-4b77-8b91-0d6db56348cb None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:38:33.030026 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-f9bd38f8-2c20-4b77-8b91-0d6db56348cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:38:33.041413 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1170/2325] 10.222.0.30 () {58 vars in 958 bytes} [Tue Sep 6 04:38:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 04:38:33.045184 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd47d1da-69d7-4995-8f9f-8df9db343786 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:38:33.057820 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd47d1da-69d7-4995-8f9f-8df9db343786 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:38:33.069026 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1265be16-59d7-4aeb-bc9d-b58ccbd49dbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:38:33.070322 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1265be16-59d7-4aeb-bc9d-b58ccbd49dbc admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:38:33.070846 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1265be16-59d7-4aeb-bc9d-b58ccbd49dbc admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:38:33.071357 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1265be16-59d7-4aeb-bc9d-b58ccbd49dbc admin admin] PATH_INFO: `/v3/domains` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:38:33.072248 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1265be16-59d7-4aeb-bc9d-b58ccbd49dbc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:38:33.072785 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1265be16-59d7-4aeb-bc9d-b58ccbd49dbc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:38:33.075911 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1265be16-59d7-4aeb-bc9d-b58ccbd49dbc admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:38:33.097973 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1156/2326] 10.222.0.30 () {58 vars in 1153 bytes} [Tue Sep 6 04:38:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:38:33.108218 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9bd38f8-2c20-4b77-8b91-0d6db56348cb None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:38:33.120407 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9bd38f8-2c20-4b77-8b91-0d6db56348cb None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:38:33.125744 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee36445-9d63-4682-bbd7-8451f6826278 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:38:33.126491 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee36445-9d63-4682-bbd7-8451f6826278 admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:38:33.126774 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee36445-9d63-4682-bbd7-8451f6826278 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:38:33.127053 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee36445-9d63-4682-bbd7-8451f6826278 admin admin] PATH_INFO: `/v3/projects` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:38:33.128745 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee36445-9d63-4682-bbd7-8451f6826278 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:38:33.130708 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee36445-9d63-4682-bbd7-8451f6826278 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:38:33.286925 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1171/2327] 10.222.0.30 () {60 vars in 1153 bytes} [Tue Sep 6 04:38:33 2022] POST /identity/v3/projects => generated 363 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 04:38:33.290721 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1265be16-59d7-4aeb-bc9d-b58ccbd49dbc admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:38:33.304363 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1265be16-59d7-4aeb-bc9d-b58ccbd49dbc admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:38:33.314506 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e16ab2-59d1-4d5b-a5f1-29228bf199a8 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:38:33.315042 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e16ab2-59d1-4d5b-a5f1-29228bf199a8 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:38:33.315042 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e16ab2-59d1-4d5b-a5f1-29228bf199a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:38:33.348076 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-03e16ab2-59d1-4d5b-a5f1-29228bf199a8 None None] MFA Rules not processed for user `65a8d2df1eed492a874ddaae3235edd0`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:38:33.349750 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-03e16ab2-59d1-4d5b-a5f1-29228bf199a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:38:33.381721 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60db394b-d029-4a14-a4b4-0f27823ed9c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:38:33.382350 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60db394b-d029-4a14-a4b4-0f27823ed9c9 admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:38:33.382628 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60db394b-d029-4a14-a4b4-0f27823ed9c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:38:33.382896 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60db394b-d029-4a14-a4b4-0f27823ed9c9 admin admin] PATH_INFO: `/v3/users` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:38:33.383544 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60db394b-d029-4a14-a4b4-0f27823ed9c9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:38:33.387547 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60db394b-d029-4a14-a4b4-0f27823ed9c9 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:38:33.457862 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1172/2328] 10.222.0.30 () {58 vars in 958 bytes} [Tue Sep 6 04:38:33 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 04:38:33.462067 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e16ab2-59d1-4d5b-a5f1-29228bf199a8 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:38:33.464304 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-03e16ab2-59d1-4d5b-a5f1-29228bf199a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:38:33.488726 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e16ab2-59d1-4d5b-a5f1-29228bf199a8 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:38:33.495487 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1157/2329] 10.222.0.30 () {60 vars in 1144 bytes} [Tue Sep 6 04:38:33 2022] POST /identity/v3/users => generated 282 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 04:38:33.495625 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c03270-4256-42c0-9c34-34f697c1df19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:38:33.495625 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c03270-4256-42c0-9c34-34f697c1df19 admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:38:33.495999 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c03270-4256-42c0-9c34-34f697c1df19 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:38:33.496408 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c03270-4256-42c0-9c34-34f697c1df19 admin admin] PATH_INFO: `/v3/projects` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:38:33.497076 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c03270-4256-42c0-9c34-34f697c1df19 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:38:33.498591 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60db394b-d029-4a14-a4b4-0f27823ed9c9 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:38:33.499361 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c03270-4256-42c0-9c34-34f697c1df19 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:38:33.516434 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60db394b-d029-4a14-a4b4-0f27823ed9c9 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:38:33.521640 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13c71d4-9fa0-4986-adc4-12b1a041d185 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:38:33.522330 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13c71d4-9fa0-4986-adc4-12b1a041d185 admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:38:33.522683 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13c71d4-9fa0-4986-adc4-12b1a041d185 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:38:33.523053 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13c71d4-9fa0-4986-adc4-12b1a041d185 admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:38:33.523703 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b13c71d4-9fa0-4986-adc4-12b1a041d185 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:38:33.525358 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b13c71d4-9fa0-4986-adc4-12b1a041d185 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:38:33.540302 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1158/2330] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Sep 6 04:38:33 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 04:38:33.544836 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13c71d4-9fa0-4986-adc4-12b1a041d185 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:38:33.556066 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13c71d4-9fa0-4986-adc4-12b1a041d185 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:38:33.561345 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5dce93-aa63-471a-a7c9-fc45d0927361 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:38:33.561688 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5dce93-aa63-471a-a7c9-fc45d0927361 admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:38:33.561994 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5dce93-aa63-471a-a7c9-fc45d0927361 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:38:33.562270 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5dce93-aa63-471a-a7c9-fc45d0927361 admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:38:33.562989 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b5dce93-aa63-471a-a7c9-fc45d0927361 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:38:33.564595 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b5dce93-aa63-471a-a7c9-fc45d0927361 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:38:33.576414 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1159/2331] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Sep 6 04:38:33 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 04:38:33.583170 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1173/2332] 10.222.0.30 () {60 vars in 1153 bytes} [Tue Sep 6 04:38:33 2022] POST /identity/v3/projects => generated 371 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 04:38:33.584111 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5dce93-aa63-471a-a7c9-fc45d0927361 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:38:33.587797 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c03270-4256-42c0-9c34-34f697c1df19 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:38:33.595596 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5dce93-aa63-471a-a7c9-fc45d0927361 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:38:33.660685 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:DELETE=7,keystone:SELECT=90,keystone:INSERT=3 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:38:33.670936 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c765473f-08e2-42c0-a12a-96be00994597 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:38:33.670936 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c765473f-08e2-42c0-a12a-96be00994597 admin admin] REQUEST_METHOD: `PUT` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:38:33.670936 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c765473f-08e2-42c0-a12a-96be00994597 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:38:33.670936 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c765473f-08e2-42c0-a12a-96be00994597 admin admin] PATH_INFO: `/v3/projects/d1c919d16efc4b7da6080eb8091654c1/users/5ff16d2bdf9745ac8c675533a3da2b23/roles/29607b2de39f40d68bb6ceef64b8a732` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:38:33.674962 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c03270-4256-42c0-9c34-34f697c1df19 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:38:33.679786 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c28cc6-43d5-4f6b-b15f-16417e07651d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:38:33.680397 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c28cc6-43d5-4f6b-b15f-16417e07651d admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:38:33.680679 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c28cc6-43d5-4f6b-b15f-16417e07651d admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:38:33.680947 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c28cc6-43d5-4f6b-b15f-16417e07651d admin admin] PATH_INFO: `/v3/users` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:38:33.681618 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c28cc6-43d5-4f6b-b15f-16417e07651d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:38:33.683659 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c28cc6-43d5-4f6b-b15f-16417e07651d admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:38:33.684734 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c765473f-08e2-42c0-a12a-96be00994597 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d1c919d16efc4b7da6080eb8091654c1, user_id=5ff16d2bdf9745ac8c675533a3da2b23, role_id=29607b2de39f40d68bb6ceef64b8a732)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:38:33.687379 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c765473f-08e2-42c0-a12a-96be00994597 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:38:33.781976 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1160/2333] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Sep 6 04:38:33 2022] PUT /identity/v3/projects/d1c919d16efc4b7da6080eb8091654c1/users/5ff16d2bdf9745ac8c675533a3da2b23/roles/29607b2de39f40d68bb6ceef64b8a732 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 04:38:33.793723 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457f894a-062d-4e0e-9ef0-3b4a0b71e8d5 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:38:33.794015 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457f894a-062d-4e0e-9ef0-3b4a0b71e8d5 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:38:33.794291 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457f894a-062d-4e0e-9ef0-3b4a0b71e8d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:38:33.798140 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1174/2334] 10.222.0.30 () {60 vars in 1144 bytes} [Tue Sep 6 04:38:33 2022] POST /identity/v3/users => generated 292 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 04:38:33.799152 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c28cc6-43d5-4f6b-b15f-16417e07651d admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:38:33.814576 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c28cc6-43d5-4f6b-b15f-16417e07651d admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:38:33.828665 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-457f894a-062d-4e0e-9ef0-3b4a0b71e8d5 None None] MFA Rules not processed for user `5ff16d2bdf9745ac8c675533a3da2b23`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:38:33.831852 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-457f894a-062d-4e0e-9ef0-3b4a0b71e8d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:38:33.882369 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:38:33.896108 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b908596e-b42f-4afb-8bc3-c889b1b400c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:38:33.899619 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b908596e-b42f-4afb-8bc3-c889b1b400c9 admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:38:33.899619 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b908596e-b42f-4afb-8bc3-c889b1b400c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:38:33.899619 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b908596e-b42f-4afb-8bc3-c889b1b400c9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:38:33.899619 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b908596e-b42f-4afb-8bc3-c889b1b400c9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:38:33.899963 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b908596e-b42f-4afb-8bc3-c889b1b400c9 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:38:33.913497 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1175/2335] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Sep 6 04:38:33 2022] GET /identity/v3/roles => generated 1420 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 04:38:33.918473 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b908596e-b42f-4afb-8bc3-c889b1b400c9 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:38:33.931685 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b908596e-b42f-4afb-8bc3-c889b1b400c9 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:38:33.937854 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec39f39-d411-49b5-9af1-1c4d2fc54831 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:38:33.938608 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec39f39-d411-49b5-9af1-1c4d2fc54831 admin admin] REQUEST_METHOD: `PUT` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:38:33.939070 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec39f39-d411-49b5-9af1-1c4d2fc54831 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:38:33.939547 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec39f39-d411-49b5-9af1-1c4d2fc54831 admin admin] PATH_INFO: `/v3/projects/88ba913e7d444dd3ac73b85d93127172/users/ce796b9a37ef4bfe9717e00200514a5d/roles/ea4c814713b149108e488de855b025e4` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:38:33.949879 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1161/2336] 10.222.0.30 () {58 vars in 958 bytes} [Tue Sep 6 04:38:33 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 04:38:33.960546 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457f894a-062d-4e0e-9ef0-3b4a0b71e8d5 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:38:33.962754 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-457f894a-062d-4e0e-9ef0-3b4a0b71e8d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:38:33.968610 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec39f39-d411-49b5-9af1-1c4d2fc54831 admin admin] RBAC: Authorizing `identity:create_grant(project_id=88ba913e7d444dd3ac73b85d93127172, user_id=ce796b9a37ef4bfe9717e00200514a5d, role_id=ea4c814713b149108e488de855b025e4)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:38:33.973102 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec39f39-d411-49b5-9af1-1c4d2fc54831 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:38:34.035400 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457f894a-062d-4e0e-9ef0-3b4a0b71e8d5 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:38:34.039324 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d192070d-ede3-478d-8ad8-45d62a8fdf71 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd09b5ab675634a2eb4b03755743f7d0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b707733ce65c42edbc1fd95ad0a1c06c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:38:34.040700 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d192070d-ede3-478d-8ad8-45d62a8fdf71 service cinder] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:38:34.041053 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d192070d-ede3-478d-8ad8-45d62a8fdf71 service cinder] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:38:34.041360 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d192070d-ede3-478d-8ad8-45d62a8fdf71 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:38:34.051196 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d192070d-ede3-478d-8ad8-45d62a8fdf71 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:38:34.052953 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d192070d-ede3-478d-8ad8-45d62a8fdf71 service cinder] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:38:34.059783 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1176/2337] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Sep 6 04:38:33 2022] PUT /identity/v3/projects/88ba913e7d444dd3ac73b85d93127172/users/ce796b9a37ef4bfe9717e00200514a5d/roles/ea4c814713b149108e488de855b025e4 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 04:38:34.063230 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1162/2338] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Sep 6 04:38:33 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 04:38:34.063849 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec39f39-d411-49b5-9af1-1c4d2fc54831 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:38:34.159734 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec39f39-d411-49b5-9af1-1c4d2fc54831 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:38:34.166781 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91abbba2-5586-4d1c-b19b-d6f9313ef4bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:38:34.167664 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91abbba2-5586-4d1c-b19b-d6f9313ef4bc admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:38:34.167946 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91abbba2-5586-4d1c-b19b-d6f9313ef4bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:38:34.168219 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91abbba2-5586-4d1c-b19b-d6f9313ef4bc admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:38:34.168881 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91abbba2-5586-4d1c-b19b-d6f9313ef4bc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:38:34.171417 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91abbba2-5586-4d1c-b19b-d6f9313ef4bc admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:38:34.189443 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d192070d-ede3-478d-8ad8-45d62a8fdf71 service cinder] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:38:34.192010 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1177/2339] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Sep 6 04:38:34 2022] GET /identity/v3/roles => generated 1420 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 04:38:34.202176 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d192070d-ede3-478d-8ad8-45d62a8fdf71 service cinder] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:38:34.206977 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d5cbb8-df98-43cb-87af-b3170e629c78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:38:34.207827 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d5cbb8-df98-43cb-87af-b3170e629c78 admin admin] REQUEST_METHOD: `PUT` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:38:34.208112 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d5cbb8-df98-43cb-87af-b3170e629c78 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:38:34.208392 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d5cbb8-df98-43cb-87af-b3170e629c78 admin admin] PATH_INFO: `/v3/projects/88ba913e7d444dd3ac73b85d93127172/users/ce796b9a37ef4bfe9717e00200514a5d/roles/ea4c814713b149108e488de855b025e4` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:38:34.210700 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d5cbb8-df98-43cb-87af-b3170e629c78 admin admin] RBAC: Authorizing `identity:create_grant(project_id=88ba913e7d444dd3ac73b85d93127172, user_id=ce796b9a37ef4bfe9717e00200514a5d, role_id=ea4c814713b149108e488de855b025e4)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:38:34.214318 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d5cbb8-df98-43cb-87af-b3170e629c78 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:38:34.232336 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1163/2340] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Sep 6 04:38:34 2022] PUT /identity/v3/projects/88ba913e7d444dd3ac73b85d93127172/users/ce796b9a37ef4bfe9717e00200514a5d/roles/ea4c814713b149108e488de855b025e4 => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 04:38:40.472126 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a8c33a-b745-4184-8ad8-63eac3a43ee2 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:38:40.472652 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a8c33a-b745-4184-8ad8-63eac3a43ee2 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:38:40.472735 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a8c33a-b745-4184-8ad8-63eac3a43ee2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:38:40.507834 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-e1a8c33a-b745-4184-8ad8-63eac3a43ee2 None None] MFA Rules not processed for user `ce796b9a37ef4bfe9717e00200514a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:38:40.509041 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-e1a8c33a-b745-4184-8ad8-63eac3a43ee2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:38:40.544905 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:38:40.581908 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1178/2341] 10.222.0.30 () {58 vars in 958 bytes} [Tue Sep 6 04:38:40 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 04:38:40.597763 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d5cbb8-df98-43cb-87af-b3170e629c78 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:38:40.600180 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-04d5cbb8-df98-43cb-87af-b3170e629c78 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:38:40.701378 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d5cbb8-df98-43cb-87af-b3170e629c78 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:38:40.707898 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406c0778-0487-4f8b-8c00-80d2ee3097e9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9db138b77afa4b068244b8ef715404fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b707733ce65c42edbc1fd95ad0a1c06c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:38:40.708534 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406c0778-0487-4f8b-8c00-80d2ee3097e9 service nova] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:38:40.708815 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406c0778-0487-4f8b-8c00-80d2ee3097e9 service nova] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:38:40.709089 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406c0778-0487-4f8b-8c00-80d2ee3097e9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:38:40.773570 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:38:40.792285 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406c0778-0487-4f8b-8c00-80d2ee3097e9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:38:40.794169 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406c0778-0487-4f8b-8c00-80d2ee3097e9 service nova] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:38:40.804346 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1164/2342] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Sep 6 04:38:40 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 207 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 04:38:43.590902 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ecd3ec-0ae9-4c29-a88d-718bfcff2e30 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:38:43.591491 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ecd3ec-0ae9-4c29-a88d-718bfcff2e30 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:38:43.591491 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ecd3ec-0ae9-4c29-a88d-718bfcff2e30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:38:43.614629 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-84ecd3ec-0ae9-4c29-a88d-718bfcff2e30 None None] MFA Rules not processed for user `65a8d2df1eed492a874ddaae3235edd0`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:38:43.615797 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-84ecd3ec-0ae9-4c29-a88d-718bfcff2e30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:38:43.622975 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1179/2343] 10.222.0.30 () {58 vars in 958 bytes} [Tue Sep 6 04:38:43 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 04:38:50.935796 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=7 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:38:53.607354 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=16 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:39:00.205529 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406c0778-0487-4f8b-8c00-80d2ee3097e9 service nova] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:39:00.206973 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-406c0778-0487-4f8b-8c00-80d2ee3097e9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:39:00.269586 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406c0778-0487-4f8b-8c00-80d2ee3097e9 service nova] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:39:00.273258 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52fa679f-28a5-4560-8c8b-517577744a74 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90492440f6a6471e8392361c098428aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b707733ce65c42edbc1fd95ad0a1c06c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:39:00.273903 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fa679f-28a5-4560-8c8b-517577744a74 service neutron] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:39:00.274183 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fa679f-28a5-4560-8c8b-517577744a74 service neutron] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:39:00.274698 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fa679f-28a5-4560-8c8b-517577744a74 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:39:00.276777 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-52fa679f-28a5-4560-8c8b-517577744a74 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:39:00.353084 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52fa679f-28a5-4560-8c8b-517577744a74 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:39:00.355452 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52fa679f-28a5-4560-8c8b-517577744a74 service neutron] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:39:00.370290 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1165/2344] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Sep 6 04:39:00 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 04:39:10.364689 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=50 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:39:19.873109 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc179c7-ce7b-471e-89f8-44eae18e3273 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:39:19.873651 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc179c7-ce7b-471e-89f8-44eae18e3273 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:39:19.873763 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc179c7-ce7b-471e-89f8-44eae18e3273 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:39:19.893645 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-efc179c7-ce7b-471e-89f8-44eae18e3273 None None] MFA Rules not processed for user `5a0294ea40914bf290902307a787642d`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:39:19.894854 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-efc179c7-ce7b-471e-89f8-44eae18e3273 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:39:19.949746 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1180/2345] 10.222.0.30 () {58 vars in 958 bytes} [Tue Sep 6 04:39:19 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 04:39:21.775351 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52fa679f-28a5-4560-8c8b-517577744a74 service neutron] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:39:21.777110 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-52fa679f-28a5-4560-8c8b-517577744a74 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:39:21.852648 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52fa679f-28a5-4560-8c8b-517577744a74 service neutron] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:39:21.858092 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9019e339-b47f-40ae-a627-27aa6488ceae service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d4f2e4dbd41441d84f03ddb30191e7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b707733ce65c42edbc1fd95ad0a1c06c', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:39:21.858681 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9019e339-b47f-40ae-a627-27aa6488ceae service placement] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:39:21.858963 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9019e339-b47f-40ae-a627-27aa6488ceae service placement] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:39:21.859298 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9019e339-b47f-40ae-a627-27aa6488ceae service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:39:21.860949 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-9019e339-b47f-40ae-a627-27aa6488ceae service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:39:21.873160 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9019e339-b47f-40ae-a627-27aa6488ceae service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:39:21.874826 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9019e339-b47f-40ae-a627-27aa6488ceae service placement] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:39:21.885040 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1166/2346] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Sep 6 04:39:21 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 04:39:29.942527 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=25 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:39:31.880683 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=32 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:39:56.557241 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd76c180-43a5-40d2-ba67-d1820e4fc071 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:39:56.557779 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd76c180-43a5-40d2-ba67-d1820e4fc071 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:39:56.557953 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd76c180-43a5-40d2-ba67-d1820e4fc071 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:39:56.585734 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-dd76c180-43a5-40d2-ba67-d1820e4fc071 None None] MFA Rules not processed for user `da5fed17101a4935bcc38b72caf9d263`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:39:56.586981 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-dd76c180-43a5-40d2-ba67-d1820e4fc071 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:39:56.659482 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1181/2347] 10.222.0.30 () {58 vars in 1009 bytes} [Tue Sep 6 04:39:56 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 04:39:56.712033 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9019e339-b47f-40ae-a627-27aa6488ceae service placement] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:39:56.712033 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-9019e339-b47f-40ae-a627-27aa6488ceae service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:39:56.807791 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9019e339-b47f-40ae-a627-27aa6488ceae service placement] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:39:56.815193 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89fc3d05-03ea-4e94-b2e7-04d6ab054208 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a676ec7f92a144edbeca8e6029ba9ee6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b707733ce65c42edbc1fd95ad0a1c06c', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:39:56.816056 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fc3d05-03ea-4e94-b2e7-04d6ab054208 service swift] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:39:56.816338 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fc3d05-03ea-4e94-b2e7-04d6ab054208 service swift] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:39:56.816799 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fc3d05-03ea-4e94-b2e7-04d6ab054208 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:39:56.833099 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89fc3d05-03ea-4e94-b2e7-04d6ab054208 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:39:56.833099 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89fc3d05-03ea-4e94-b2e7-04d6ab054208 service swift] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:39:56.842941 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1167/2348] 10.222.0.30 () {60 vars in 1384 bytes} [Tue Sep 6 04:39:56 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 06 04:39:59.265440 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd76c180-43a5-40d2-ba67-d1820e4fc071 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:39:59.370916 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd76c180-43a5-40d2-ba67-d1820e4fc071 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:39:59.376475 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799058e2-41ec-4e98-bba3-3537f2ca2ecc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:39:59.377475 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799058e2-41ec-4e98-bba3-3537f2ca2ecc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:39:59.379708 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799058e2-41ec-4e98-bba3-3537f2ca2ecc admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:39:59.379708 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799058e2-41ec-4e98-bba3-3537f2ca2ecc admin admin] PATH_INFO: `/v3/users/5ff16d2bdf9745ac8c675533a3da2b23` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:39:59.379708 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799058e2-41ec-4e98-bba3-3537f2ca2ecc admin admin] RBAC: Authorizing `identity:delete_user(user_id=5ff16d2bdf9745ac8c675533a3da2b23)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:39:59.386977 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799058e2-41ec-4e98-bba3-3537f2ca2ecc admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:39:59.642671 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-799058e2-41ec-4e98-bba3-3537f2ca2ecc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5ff16d2bdf9745ac8c675533a3da2b23'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:39:59.854793 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-799058e2-41ec-4e98-bba3-3537f2ca2ecc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5ff16d2bdf9745ac8c675533a3da2b23'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:39:59.855664 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-799058e2-41ec-4e98-bba3-3537f2ca2ecc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5ff16d2bdf9745ac8c675533a3da2b23'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:39:59.884239 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-799058e2-41ec-4e98-bba3-3537f2ca2ecc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5ff16d2bdf9745ac8c675533a3da2b23'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:39:59.893445 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-799058e2-41ec-4e98-bba3-3537f2ca2ecc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5ff16d2bdf9745ac8c675533a3da2b23'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:39:59.900058 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1182/2349] 10.222.0.30 () {58 vars in 1224 bytes} [Tue Sep 6 04:39:59 2022] DELETE /identity/v3/users/5ff16d2bdf9745ac8c675533a3da2b23 => generated 0 bytes in 635 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 04:39:59.905056 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37786a6-58f8-451d-a91f-039371ff5f38 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:39:59.905775 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37786a6-58f8-451d-a91f-039371ff5f38 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:39:59.905775 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37786a6-58f8-451d-a91f-039371ff5f38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:39:59.927787 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-b37786a6-58f8-451d-a91f-039371ff5f38 None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:39:59.927787 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-b37786a6-58f8-451d-a91f-039371ff5f38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:40:00.000624 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1168/2350] 10.222.0.30 () {58 vars in 958 bytes} [Tue Sep 6 04:39:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 04:40:00.012807 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799058e2-41ec-4e98-bba3-3537f2ca2ecc admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:40:00.014705 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-799058e2-41ec-4e98-bba3-3537f2ca2ecc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:40:00.047946 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:40:00.080944 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799058e2-41ec-4e98-bba3-3537f2ca2ecc admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:40:00.085122 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f8d8d7-632e-4cd7-9a57-bed1f27171e0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90492440f6a6471e8392361c098428aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b707733ce65c42edbc1fd95ad0a1c06c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:40:00.085917 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f8d8d7-632e-4cd7-9a57-bed1f27171e0 service neutron] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:40:00.086302 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f8d8d7-632e-4cd7-9a57-bed1f27171e0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:40:00.086824 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f8d8d7-632e-4cd7-9a57-bed1f27171e0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:40:00.104925 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f8d8d7-632e-4cd7-9a57-bed1f27171e0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:40:00.109069 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f8d8d7-632e-4cd7-9a57-bed1f27171e0 service neutron] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:40:00.120502 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1183/2351] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Sep 6 04:40:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 04:40:00.594803 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37786a6-58f8-451d-a91f-039371ff5f38 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:40:00.596856 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-b37786a6-58f8-451d-a91f-039371ff5f38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:40:00.621527 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37786a6-58f8-451d-a91f-039371ff5f38 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:40:00.633415 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef0d763-9a9c-4ce3-999b-621ae720c055 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:40:00.641937 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef0d763-9a9c-4ce3-999b-621ae720c055 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:40:00.647259 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef0d763-9a9c-4ce3-999b-621ae720c055 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:40:00.647788 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef0d763-9a9c-4ce3-999b-621ae720c055 admin admin] PATH_INFO: `/v3/projects/d1c919d16efc4b7da6080eb8091654c1` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:40:00.652072 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef0d763-9a9c-4ce3-999b-621ae720c055 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d1c919d16efc4b7da6080eb8091654c1)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:40:00.654104 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef0d763-9a9c-4ce3-999b-621ae720c055 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:40:00.818947 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-1ef0d763-9a9c-4ce3-999b-621ae720c055 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd1c919d16efc4b7da6080eb8091654c1', 'request_id': 'req-1ef0d763-9a9c-4ce3-999b-621ae720c055'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:40:00.890891 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-1ef0d763-9a9c-4ce3-999b-621ae720c055 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd1c919d16efc4b7da6080eb8091654c1', 'request_id': 'req-1ef0d763-9a9c-4ce3-999b-621ae720c055'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:40:00.895823 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-1ef0d763-9a9c-4ce3-999b-621ae720c055 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd1c919d16efc4b7da6080eb8091654c1', 'request_id': 'req-1ef0d763-9a9c-4ce3-999b-621ae720c055'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:40:00.914624 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-1ef0d763-9a9c-4ce3-999b-621ae720c055 admin admin] The token cache is being invalidate because project d1c919d16efc4b7da6080eb8091654c1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81697) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 04:40:00.915375 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-1ef0d763-9a9c-4ce3-999b-621ae720c055 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:40:00.917913 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-1ef0d763-9a9c-4ce3-999b-621ae720c055 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:40:00.919865 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1169/2352] 10.222.0.30 () {58 vars in 1233 bytes} [Tue Sep 6 04:40:00 2022] DELETE /identity/v3/projects/d1c919d16efc4b7da6080eb8091654c1 => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 04:40:10.187969 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=15 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:40:10.907455 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=86,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:40:11.842177 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f8d8d7-632e-4cd7-9a57-bed1f27171e0 service neutron] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:40:11.844127 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-55f8d8d7-632e-4cd7-9a57-bed1f27171e0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:40:11.855363 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f8d8d7-632e-4cd7-9a57-bed1f27171e0 service neutron] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:40:11.904622 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758ee5bc-01b4-469b-b4b7-cf6ba910c711 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90492440f6a6471e8392361c098428aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b707733ce65c42edbc1fd95ad0a1c06c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:40:11.905375 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758ee5bc-01b4-469b-b4b7-cf6ba910c711 service neutron] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:40:11.906043 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758ee5bc-01b4-469b-b4b7-cf6ba910c711 service neutron] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:40:11.906327 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758ee5bc-01b4-469b-b4b7-cf6ba910c711 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:40:11.908055 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-758ee5bc-01b4-469b-b4b7-cf6ba910c711 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:40:11.970696 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758ee5bc-01b4-469b-b4b7-cf6ba910c711 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:40:11.972073 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758ee5bc-01b4-469b-b4b7-cf6ba910c711 service neutron] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:40:11.981492 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1184/2353] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Sep 6 04:40:11 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 04:40:16.554133 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6aa089-3361-4b80-b98d-53864e123d00 None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:40:16.554727 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6aa089-3361-4b80-b98d-53864e123d00 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:40:16.554727 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6aa089-3361-4b80-b98d-53864e123d00 None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:40:16.556014 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1170/2354] 10.222.0.30 () {54 vars in 921 bytes} [Tue Sep 6 04:40:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 04:40:16.562781 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4ff636-a761-4d9e-85df-d8adbfb296a4 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:40:16.563267 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4ff636-a761-4d9e-85df-d8adbfb296a4 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:40:16.563384 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4ff636-a761-4d9e-85df-d8adbfb296a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:40:16.583296 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-ef4ff636-a761-4d9e-85df-d8adbfb296a4 None None] MFA Rules not processed for user `9db138b77afa4b068244b8ef715404fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:40:16.584497 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-ef4ff636-a761-4d9e-85df-d8adbfb296a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:40:16.653514 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1185/2355] 10.222.0.30 () {58 vars in 1018 bytes} [Tue Sep 6 04:40:16 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 04:40:16.664799 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6aa089-3361-4b80-b98d-53864e123d00 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:40:16.666525 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-de6aa089-3361-4b80-b98d-53864e123d00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:40:16.724983 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6aa089-3361-4b80-b98d-53864e123d00 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:40:16.728570 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2266313-b73f-4e21-a030-1111db79614c service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae5c31fb1d0342c0937bbc83a415113d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b707733ce65c42edbc1fd95ad0a1c06c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:40:16.729163 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2266313-b73f-4e21-a030-1111db79614c service glance] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:40:16.729471 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2266313-b73f-4e21-a030-1111db79614c service glance] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:40:16.729742 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2266313-b73f-4e21-a030-1111db79614c service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:40:16.740418 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2266313-b73f-4e21-a030-1111db79614c service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:40:16.741861 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2266313-b73f-4e21-a030-1111db79614c service glance] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:40:16.753343 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1171/2356] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Sep 6 04:40:16 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 04:40:16.987861 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee16ab4-99bf-478c-917b-c022c018352f None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:40:16.988077 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee16ab4-99bf-478c-917b-c022c018352f None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:40:16.988359 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee16ab4-99bf-478c-917b-c022c018352f None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:40:16.989762 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1186/2357] 10.222.0.30 () {54 vars in 921 bytes} [Tue Sep 6 04:40:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 04:40:16.994645 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c2e634-2434-4e4e-a820-1bd03e0c52cc None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:40:16.994820 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c2e634-2434-4e4e-a820-1bd03e0c52cc None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:40:16.995123 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c2e634-2434-4e4e-a820-1bd03e0c52cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:40:17.014675 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-00c2e634-2434-4e4e-a820-1bd03e0c52cc None None] MFA Rules not processed for user `90492440f6a6471e8392361c098428aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:40:17.015889 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-00c2e634-2434-4e4e-a820-1bd03e0c52cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:40:17.022324 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1172/2358] 10.222.0.30 () {58 vars in 1018 bytes} [Tue Sep 6 04:40:16 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 04:40:17.034730 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee16ab4-99bf-478c-917b-c022c018352f None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:40:17.036442 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-aee16ab4-99bf-478c-917b-c022c018352f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:40:17.050253 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee16ab4-99bf-478c-917b-c022c018352f None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:40:17.053781 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ea797a-033b-4aea-9779-cc0642f77c9b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90492440f6a6471e8392361c098428aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b707733ce65c42edbc1fd95ad0a1c06c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:40:17.054369 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ea797a-033b-4aea-9779-cc0642f77c9b service neutron] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:40:17.054648 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ea797a-033b-4aea-9779-cc0642f77c9b service neutron] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:40:17.054920 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ea797a-033b-4aea-9779-cc0642f77c9b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:40:17.062761 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ea797a-033b-4aea-9779-cc0642f77c9b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:40:17.064196 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ea797a-033b-4aea-9779-cc0642f77c9b service neutron] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:40:17.072179 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1187/2359] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Sep 6 04:40:17 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 04:40:17.166782 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d873ee-05c5-43f7-a5c9-babd44abc404 None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:40:17.167093 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d873ee-05c5-43f7-a5c9-babd44abc404 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:40:17.167372 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d873ee-05c5-43f7-a5c9-babd44abc404 None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:40:17.168741 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1173/2360] 10.222.0.30 () {54 vars in 921 bytes} [Tue Sep 6 04:40:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 04:40:17.175009 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90d9183-da9e-411b-a46d-82a9e348f3dd None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:40:17.175299 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90d9183-da9e-411b-a46d-82a9e348f3dd None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:40:17.175578 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90d9183-da9e-411b-a46d-82a9e348f3dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:40:17.197201 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-c90d9183-da9e-411b-a46d-82a9e348f3dd None None] MFA Rules not processed for user `9db138b77afa4b068244b8ef715404fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:40:17.198423 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-c90d9183-da9e-411b-a46d-82a9e348f3dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:40:17.209331 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1188/2361] 10.222.0.30 () {58 vars in 1018 bytes} [Tue Sep 6 04:40:17 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 04:40:17.220217 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d873ee-05c5-43f7-a5c9-babd44abc404 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:40:17.221904 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-74d873ee-05c5-43f7-a5c9-babd44abc404 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:40:17.237070 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d873ee-05c5-43f7-a5c9-babd44abc404 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:40:17.241606 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ceade5-d07b-4dbc-993c-0b44e1211b3e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90492440f6a6471e8392361c098428aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b707733ce65c42edbc1fd95ad0a1c06c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:40:17.244198 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ceade5-d07b-4dbc-993c-0b44e1211b3e service neutron] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:40:17.244198 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ceade5-d07b-4dbc-993c-0b44e1211b3e service neutron] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:40:17.244198 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ceade5-d07b-4dbc-993c-0b44e1211b3e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:40:17.254116 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ceade5-d07b-4dbc-993c-0b44e1211b3e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:40:17.255543 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ceade5-d07b-4dbc-993c-0b44e1211b3e service neutron] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:40:17.266366 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1174/2362] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Sep 6 04:40:17 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 04:40:18.881272 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90d9183-da9e-411b-a46d-82a9e348f3dd None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:40:18.891281 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90d9183-da9e-411b-a46d-82a9e348f3dd None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:40:18.894797 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275d0644-fd3c-4e18-a4f8-2a5ce9aee32e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90492440f6a6471e8392361c098428aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b707733ce65c42edbc1fd95ad0a1c06c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:40:18.895387 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275d0644-fd3c-4e18-a4f8-2a5ce9aee32e service neutron] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:40:18.895663 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275d0644-fd3c-4e18-a4f8-2a5ce9aee32e service neutron] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:40:18.895929 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275d0644-fd3c-4e18-a4f8-2a5ce9aee32e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:40:18.897635 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-275d0644-fd3c-4e18-a4f8-2a5ce9aee32e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:40:18.912474 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275d0644-fd3c-4e18-a4f8-2a5ce9aee32e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:40:18.913938 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275d0644-fd3c-4e18-a4f8-2a5ce9aee32e service neutron] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:40:18.924595 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1189/2363] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Sep 6 04:40:18 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 04:40:22.479681 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ceade5-d07b-4dbc-993c-0b44e1211b3e service neutron] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:40:22.488443 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ceade5-d07b-4dbc-993c-0b44e1211b3e service neutron] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:40:22.492072 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c97404b-52a3-4cb5-b49d-9861aad9ec47 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90492440f6a6471e8392361c098428aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b707733ce65c42edbc1fd95ad0a1c06c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:40:22.492668 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c97404b-52a3-4cb5-b49d-9861aad9ec47 service neutron] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:40:22.492943 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c97404b-52a3-4cb5-b49d-9861aad9ec47 service neutron] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:40:22.493210 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c97404b-52a3-4cb5-b49d-9861aad9ec47 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:40:22.495035 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-7c97404b-52a3-4cb5-b49d-9861aad9ec47 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:40:22.508539 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c97404b-52a3-4cb5-b49d-9861aad9ec47 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:40:22.509995 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c97404b-52a3-4cb5-b49d-9861aad9ec47 service neutron] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:40:22.521728 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1175/2364] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Sep 6 04:40:22 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 04:40:26.571610 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6bc845-dbc4-455a-bc83-bbc93a28d67e None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:40:26.572084 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6bc845-dbc4-455a-bc83-bbc93a28d67e None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:40:26.572176 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6bc845-dbc4-455a-bc83-bbc93a28d67e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:40:26.591587 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-da6bc845-dbc4-455a-bc83-bbc93a28d67e None None] MFA Rules not processed for user `da5fed17101a4935bcc38b72caf9d263`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:40:26.592775 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-da6bc845-dbc4-455a-bc83-bbc93a28d67e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:40:26.596993 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:40:26.649226 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1190/2365] 10.222.0.30 () {58 vars in 1009 bytes} [Tue Sep 6 04:40:26 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 04:40:26.661880 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c97404b-52a3-4cb5-b49d-9861aad9ec47 service neutron] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:40:26.663292 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-7c97404b-52a3-4cb5-b49d-9861aad9ec47 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:40:26.732490 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c97404b-52a3-4cb5-b49d-9861aad9ec47 service neutron] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:40:26.737694 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2978efff-60c6-4134-b010-da4f1a0f0731 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a676ec7f92a144edbeca8e6029ba9ee6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b707733ce65c42edbc1fd95ad0a1c06c', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:40:26.738342 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2978efff-60c6-4134-b010-da4f1a0f0731 service swift] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:40:26.738625 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2978efff-60c6-4134-b010-da4f1a0f0731 service swift] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:40:26.738899 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2978efff-60c6-4134-b010-da4f1a0f0731 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:40:26.748586 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2978efff-60c6-4134-b010-da4f1a0f0731 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:40:26.750206 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2978efff-60c6-4134-b010-da4f1a0f0731 service swift] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:40:26.759035 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1176/2366] 10.222.0.30 () {60 vars in 1384 bytes} [Tue Sep 6 04:40:26 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 06 04:40:27.235908 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd83130-dc82-4b34-92e5-44a54d223987 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:40:27.236194 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd83130-dc82-4b34-92e5-44a54d223987 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:40:27.236467 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd83130-dc82-4b34-92e5-44a54d223987 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:40:27.261436 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-6dd83130-dc82-4b34-92e5-44a54d223987 None None] MFA Rules not processed for user `784815b8c6d0400c92caa54981b137d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:40:27.262717 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-6dd83130-dc82-4b34-92e5-44a54d223987 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:40:27.322381 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1191/2367] 10.222.0.30 () {58 vars in 958 bytes} [Tue Sep 6 04:40:27 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 04:40:31.076168 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2978efff-60c6-4134-b010-da4f1a0f0731 service swift] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:40:31.077708 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-2978efff-60c6-4134-b010-da4f1a0f0731 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:40:31.095693 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2978efff-60c6-4134-b010-da4f1a0f0731 service swift] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:40:31.101411 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285e0c66-46ae-407a-b8cb-257f85d35942 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9db138b77afa4b068244b8ef715404fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b707733ce65c42edbc1fd95ad0a1c06c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:40:31.102032 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285e0c66-46ae-407a-b8cb-257f85d35942 service nova] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:40:31.102307 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285e0c66-46ae-407a-b8cb-257f85d35942 service nova] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:40:31.102579 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285e0c66-46ae-407a-b8cb-257f85d35942 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:40:31.104549 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-285e0c66-46ae-407a-b8cb-257f85d35942 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:40:31.169276 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285e0c66-46ae-407a-b8cb-257f85d35942 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:40:31.171105 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285e0c66-46ae-407a-b8cb-257f85d35942 service nova] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:40:31.178118 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:40:31.182427 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1177/2368] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Sep 6 04:40:31 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 04:40:37.315553 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=46 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:40:41.211941 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:40:45.912528 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd83130-dc82-4b34-92e5-44a54d223987 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:40:45.923329 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd83130-dc82-4b34-92e5-44a54d223987 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:40:45.926992 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2fef93-5388-47cf-a297-30e3e962daab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90492440f6a6471e8392361c098428aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b707733ce65c42edbc1fd95ad0a1c06c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:40:45.927598 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2fef93-5388-47cf-a297-30e3e962daab service neutron] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:40:45.927874 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2fef93-5388-47cf-a297-30e3e962daab service neutron] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:40:45.928144 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2fef93-5388-47cf-a297-30e3e962daab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:40:45.929959 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-1b2fef93-5388-47cf-a297-30e3e962daab service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:40:45.940097 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2fef93-5388-47cf-a297-30e3e962daab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:40:45.941536 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2fef93-5388-47cf-a297-30e3e962daab service neutron] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:40:45.949807 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1192/2369] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Sep 6 04:40:45 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 04:40:55.946232 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=6 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:41:01.315066 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285e0c66-46ae-407a-b8cb-257f85d35942 service nova] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:41:01.316582 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-285e0c66-46ae-407a-b8cb-257f85d35942 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:41:01.331093 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285e0c66-46ae-407a-b8cb-257f85d35942 service nova] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:41:01.336593 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8469871-fba4-482d-a509-12118373ecc4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9db138b77afa4b068244b8ef715404fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b707733ce65c42edbc1fd95ad0a1c06c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:41:01.337198 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8469871-fba4-482d-a509-12118373ecc4 service nova] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:41:01.342531 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8469871-fba4-482d-a509-12118373ecc4 service nova] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:41:01.342531 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8469871-fba4-482d-a509-12118373ecc4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:41:01.342531 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-d8469871-fba4-482d-a509-12118373ecc4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:41:01.400659 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8469871-fba4-482d-a509-12118373ecc4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:41:01.402496 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8469871-fba4-482d-a509-12118373ecc4 service nova] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:41:01.412343 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1178/2370] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Sep 6 04:41:01 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 04:41:05.263783 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e607a4-3492-476d-aefa-4d9529433b87 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:41:05.264287 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e607a4-3492-476d-aefa-4d9529433b87 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:41:05.264384 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e607a4-3492-476d-aefa-4d9529433b87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:41:05.286063 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-22e607a4-3492-476d-aefa-4d9529433b87 None None] MFA Rules not processed for user `da5fed17101a4935bcc38b72caf9d263`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:41:05.287275 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-22e607a4-3492-476d-aefa-4d9529433b87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:41:05.294740 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1193/2371] 10.222.0.30 () {58 vars in 967 bytes} [Tue Sep 6 04:41:05 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 04:41:05.316826 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8469871-fba4-482d-a509-12118373ecc4 service nova] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:41:05.328221 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8469871-fba4-482d-a509-12118373ecc4 service nova] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:41:05.333088 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b6764e-06d8-427d-ad20-55d7fd15d91d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a676ec7f92a144edbeca8e6029ba9ee6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b707733ce65c42edbc1fd95ad0a1c06c', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:41:05.333745 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b6764e-06d8-427d-ad20-55d7fd15d91d service swift] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:41:05.334024 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b6764e-06d8-427d-ad20-55d7fd15d91d service swift] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:41:05.334297 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b6764e-06d8-427d-ad20-55d7fd15d91d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:41:05.342697 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b6764e-06d8-427d-ad20-55d7fd15d91d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:41:05.344246 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b6764e-06d8-427d-ad20-55d7fd15d91d service swift] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:41:05.352154 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1179/2372] 10.222.0.30 () {60 vars in 1384 bytes} [Tue Sep 6 04:41:05 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 06 04:41:15.280757 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=3 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:41:15.348946 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=34 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:41:39.935741 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c7dbc4-7d10-4919-acc1-304316856f27 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:41:39.936297 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c7dbc4-7d10-4919-acc1-304316856f27 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:41:39.936494 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c7dbc4-7d10-4919-acc1-304316856f27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:41:39.956417 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-55c7dbc4-7d10-4919-acc1-304316856f27 None None] MFA Rules not processed for user `dec1c17550b849dbadbc16e98530b455`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:41:39.957727 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-55c7dbc4-7d10-4919-acc1-304316856f27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:41:39.964552 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1194/2373] 10.222.0.30 () {58 vars in 958 bytes} [Tue Sep 6 04:41:39 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 04:41:45.763545 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b6764e-06d8-427d-ad20-55d7fd15d91d service swift] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:41:45.775811 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b6764e-06d8-427d-ad20-55d7fd15d91d service swift] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:41:45.782174 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c221306b-52da-4515-b75d-4c647e9da4d7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9db138b77afa4b068244b8ef715404fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b707733ce65c42edbc1fd95ad0a1c06c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:41:45.782776 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c221306b-52da-4515-b75d-4c647e9da4d7 service nova] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:41:45.783059 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c221306b-52da-4515-b75d-4c647e9da4d7 service nova] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:41:45.783335 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c221306b-52da-4515-b75d-4c647e9da4d7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:41:45.785047 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-c221306b-52da-4515-b75d-4c647e9da4d7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:41:45.797581 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c221306b-52da-4515-b75d-4c647e9da4d7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:41:45.799150 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c221306b-52da-4515-b75d-4c647e9da4d7 service nova] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:41:45.809846 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1180/2374] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Sep 6 04:41:45 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 04:41:48.603936 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c7dbc4-7d10-4919-acc1-304316856f27 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:41:48.618161 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c7dbc4-7d10-4919-acc1-304316856f27 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:41:48.623978 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2787af78-4a9e-4650-9bf7-631b28712d54 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9db138b77afa4b068244b8ef715404fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b707733ce65c42edbc1fd95ad0a1c06c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:41:48.625101 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2787af78-4a9e-4650-9bf7-631b28712d54 service nova] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:41:48.625589 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2787af78-4a9e-4650-9bf7-631b28712d54 service nova] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:41:48.626019 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2787af78-4a9e-4650-9bf7-631b28712d54 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:41:48.630906 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-2787af78-4a9e-4650-9bf7-631b28712d54 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:41:48.644783 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2787af78-4a9e-4650-9bf7-631b28712d54 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:41:48.646471 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2787af78-4a9e-4650-9bf7-631b28712d54 service nova] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:41:48.658606 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1195/2375] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Sep 6 04:41:48 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 04:41:55.805234 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=6 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:41:58.652978 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=9 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:42:02.304827 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c221306b-52da-4515-b75d-4c647e9da4d7 service nova] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:42:02.316888 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c221306b-52da-4515-b75d-4c647e9da4d7 service nova] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:42:02.322392 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7965963-0d9d-4145-9b5f-7474a5dff797 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9db138b77afa4b068244b8ef715404fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b707733ce65c42edbc1fd95ad0a1c06c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:42:02.322996 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7965963-0d9d-4145-9b5f-7474a5dff797 service nova] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:42:02.323279 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7965963-0d9d-4145-9b5f-7474a5dff797 service nova] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:42:02.323550 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7965963-0d9d-4145-9b5f-7474a5dff797 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:42:02.325279 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-a7965963-0d9d-4145-9b5f-7474a5dff797 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:42:02.394045 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7965963-0d9d-4145-9b5f-7474a5dff797 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:42:02.395658 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7965963-0d9d-4145-9b5f-7474a5dff797 service nova] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:42:02.405752 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1181/2376] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Sep 6 04:42:02 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 04:42:12.400974 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=28 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:42:19.603179 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33c1daa-d087-4c74-800d-c7b1a807242c None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:42:19.603715 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33c1daa-d087-4c74-800d-c7b1a807242c None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:42:19.603715 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33c1daa-d087-4c74-800d-c7b1a807242c None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:42:19.605125 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1196/2377] 10.222.0.30 () {54 vars in 921 bytes} [Tue Sep 6 04:42:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 04:42:19.634932 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f43a0b-1427-4433-a2d5-34f5ad46c065 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:42:19.635452 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f43a0b-1427-4433-a2d5-34f5ad46c065 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:42:19.635612 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f43a0b-1427-4433-a2d5-34f5ad46c065 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:42:19.663917 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-28f43a0b-1427-4433-a2d5-34f5ad46c065 None None] MFA Rules not processed for user `1d4f2e4dbd41441d84f03ddb30191e7a`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:42:19.665145 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-28f43a0b-1427-4433-a2d5-34f5ad46c065 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:42:19.731678 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1182/2378] 10.222.0.30 () {58 vars in 1018 bytes} [Tue Sep 6 04:42:19 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 04:42:19.745871 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33c1daa-d087-4c74-800d-c7b1a807242c None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:42:19.747478 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-e33c1daa-d087-4c74-800d-c7b1a807242c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:42:19.753953 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f43a0b-1427-4433-a2d5-34f5ad46c065 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:42:19.755610 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-28f43a0b-1427-4433-a2d5-34f5ad46c065 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:42:19.767482 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33c1daa-d087-4c74-800d-c7b1a807242c None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:42:19.773132 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f046e5-25d8-4035-97c5-86e2d6b37e60 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d4f2e4dbd41441d84f03ddb30191e7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b707733ce65c42edbc1fd95ad0a1c06c', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:42:19.773873 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f046e5-25d8-4035-97c5-86e2d6b37e60 service placement] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:42:19.774154 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f046e5-25d8-4035-97c5-86e2d6b37e60 service placement] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:42:19.774422 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f046e5-25d8-4035-97c5-86e2d6b37e60 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:42:19.783779 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f046e5-25d8-4035-97c5-86e2d6b37e60 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:42:19.785514 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f046e5-25d8-4035-97c5-86e2d6b37e60 service placement] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:42:19.796869 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1197/2379] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Sep 6 04:42:19 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 04:42:19.829920 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f43a0b-1427-4433-a2d5-34f5ad46c065 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:42:19.835831 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c10b43-5f28-4196-b841-b4f6c6fb5feb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:42:19.836456 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c10b43-5f28-4196-b841-b4f6c6fb5feb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:42:19.836776 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c10b43-5f28-4196-b841-b4f6c6fb5feb admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:42:19.837046 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c10b43-5f28-4196-b841-b4f6c6fb5feb admin admin] PATH_INFO: `/v3/users/784815b8c6d0400c92caa54981b137d9` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:42:19.838219 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c10b43-5f28-4196-b841-b4f6c6fb5feb admin admin] RBAC: Authorizing `identity:delete_user(user_id=784815b8c6d0400c92caa54981b137d9)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:42:19.840226 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c10b43-5f28-4196-b841-b4f6c6fb5feb admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:42:20.012971 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-98c10b43-5f28-4196-b841-b4f6c6fb5feb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '784815b8c6d0400c92caa54981b137d9'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:42:20.067046 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-98c10b43-5f28-4196-b841-b4f6c6fb5feb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '784815b8c6d0400c92caa54981b137d9'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:42:20.067704 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-98c10b43-5f28-4196-b841-b4f6c6fb5feb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '784815b8c6d0400c92caa54981b137d9'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:42:20.075582 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-98c10b43-5f28-4196-b841-b4f6c6fb5feb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '784815b8c6d0400c92caa54981b137d9'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:42:20.088128 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-98c10b43-5f28-4196-b841-b4f6c6fb5feb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '784815b8c6d0400c92caa54981b137d9'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:42:20.093990 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1183/2380] 10.222.0.30 () {58 vars in 1224 bytes} [Tue Sep 6 04:42:19 2022] DELETE /identity/v3/users/784815b8c6d0400c92caa54981b137d9 => generated 0 bytes in 341 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 04:42:20.342699 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f046e5-25d8-4035-97c5-86e2d6b37e60 service placement] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:42:20.357990 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f046e5-25d8-4035-97c5-86e2d6b37e60 service placement] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:42:20.424068 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae02c026-f035-4039-b937-baf354caed60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:42:20.424971 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae02c026-f035-4039-b937-baf354caed60 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:42:20.424971 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae02c026-f035-4039-b937-baf354caed60 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:42:20.425237 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae02c026-f035-4039-b937-baf354caed60 admin admin] PATH_INFO: `/v3/projects/36495849f5124872ba74fe21b00f72fe` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:42:20.426438 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae02c026-f035-4039-b937-baf354caed60 admin admin] RBAC: Authorizing `identity:delete_project(project_id=36495849f5124872ba74fe21b00f72fe)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:42:20.428362 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae02c026-f035-4039-b937-baf354caed60 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:42:20.562593 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-ae02c026-f035-4039-b937-baf354caed60 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '36495849f5124872ba74fe21b00f72fe', 'request_id': 'req-ae02c026-f035-4039-b937-baf354caed60'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:42:20.568713 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-ae02c026-f035-4039-b937-baf354caed60 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '36495849f5124872ba74fe21b00f72fe', 'request_id': 'req-ae02c026-f035-4039-b937-baf354caed60'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:42:20.650794 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-ae02c026-f035-4039-b937-baf354caed60 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '36495849f5124872ba74fe21b00f72fe', 'request_id': 'req-ae02c026-f035-4039-b937-baf354caed60'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:42:20.664759 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-ae02c026-f035-4039-b937-baf354caed60 admin admin] The token cache is being invalidate because project 36495849f5124872ba74fe21b00f72fe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81698) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 04:42:20.665052 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-ae02c026-f035-4039-b937-baf354caed60 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:42:20.665782 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-ae02c026-f035-4039-b937-baf354caed60 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:42:20.667400 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1198/2381] 10.222.0.30 () {58 vars in 1233 bytes} [Tue Sep 6 04:42:20 2022] DELETE /identity/v3/projects/36495849f5124872ba74fe21b00f72fe => generated 0 bytes in 325 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 04:42:20.672449 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c01c31e-942a-46fb-9695-8df35955728c None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:42:20.674133 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c01c31e-942a-46fb-9695-8df35955728c None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:42:20.674495 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c01c31e-942a-46fb-9695-8df35955728c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:42:20.694502 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-8c01c31e-942a-46fb-9695-8df35955728c None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:42:20.695736 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-8c01c31e-942a-46fb-9695-8df35955728c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:42:20.705967 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:42:20.765943 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1184/2382] 10.222.0.30 () {58 vars in 958 bytes} [Tue Sep 6 04:42:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 04:42:20.774659 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6bef1b-91ae-4fe4-9834-d31015cb40f9 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:42:20.774953 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6bef1b-91ae-4fe4-9834-d31015cb40f9 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:42:20.775237 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6bef1b-91ae-4fe4-9834-d31015cb40f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:42:20.797122 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-4d6bef1b-91ae-4fe4-9834-d31015cb40f9 None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:42:20.798433 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-4d6bef1b-91ae-4fe4-9834-d31015cb40f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:42:20.809463 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1199/2383] 10.222.0.30 () {58 vars in 958 bytes} [Tue Sep 6 04:42:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 04:42:20.811439 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c01c31e-942a-46fb-9695-8df35955728c None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:42:20.822403 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c01c31e-942a-46fb-9695-8df35955728c None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:42:20.827100 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c5dc61-1c4d-4a3e-b5e3-baa8a97ddfd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:42:20.827711 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c5dc61-1c4d-4a3e-b5e3-baa8a97ddfd7 admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:42:20.827996 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c5dc61-1c4d-4a3e-b5e3-baa8a97ddfd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:42:20.828272 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c5dc61-1c4d-4a3e-b5e3-baa8a97ddfd7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:42:20.828917 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c5dc61-1c4d-4a3e-b5e3-baa8a97ddfd7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:42:20.829295 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c5dc61-1c4d-4a3e-b5e3-baa8a97ddfd7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:42:20.831022 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c5dc61-1c4d-4a3e-b5e3-baa8a97ddfd7 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:42:20.844824 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1185/2384] 10.222.0.30 () {58 vars in 1153 bytes} [Tue Sep 6 04:42:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 04:42:20.849460 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6bef1b-91ae-4fe4-9834-d31015cb40f9 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:42:20.859153 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6bef1b-91ae-4fe4-9834-d31015cb40f9 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:42:20.867664 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64eb765c-7abf-450b-ae8f-1c17836eac0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:42:20.868292 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64eb765c-7abf-450b-ae8f-1c17836eac0b admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:42:20.868562 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64eb765c-7abf-450b-ae8f-1c17836eac0b admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:42:20.868838 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64eb765c-7abf-450b-ae8f-1c17836eac0b admin admin] PATH_INFO: `/v3/projects` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:42:20.869636 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64eb765c-7abf-450b-ae8f-1c17836eac0b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:42:20.871637 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64eb765c-7abf-450b-ae8f-1c17836eac0b admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:42:20.956998 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1200/2385] 10.222.0.30 () {60 vars in 1153 bytes} [Tue Sep 6 04:42:20 2022] POST /identity/v3/projects => generated 397 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 04:42:20.958291 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c5dc61-1c4d-4a3e-b5e3-baa8a97ddfd7 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:42:20.969081 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c5dc61-1c4d-4a3e-b5e3-baa8a97ddfd7 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:42:21.026704 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9283f5b1-4e19-47c4-90d5-9e11d4dff51d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:42:21.027318 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9283f5b1-4e19-47c4-90d5-9e11d4dff51d admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:42:21.027869 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9283f5b1-4e19-47c4-90d5-9e11d4dff51d admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:42:21.028139 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9283f5b1-4e19-47c4-90d5-9e11d4dff51d admin admin] PATH_INFO: `/v3/users` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:42:21.028788 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9283f5b1-4e19-47c4-90d5-9e11d4dff51d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:42:21.030859 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9283f5b1-4e19-47c4-90d5-9e11d4dff51d admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:42:21.127074 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1186/2386] 10.222.0.30 () {60 vars in 1144 bytes} [Tue Sep 6 04:42:20 2022] POST /identity/v3/users => generated 299 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 04:42:21.130293 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64eb765c-7abf-450b-ae8f-1c17836eac0b admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:42:21.138710 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64eb765c-7abf-450b-ae8f-1c17836eac0b admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:42:21.144091 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf48d5d-7e0f-42b4-b446-166f96ba17e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:42:21.144694 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf48d5d-7e0f-42b4-b446-166f96ba17e5 admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:42:21.144970 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf48d5d-7e0f-42b4-b446-166f96ba17e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:42:21.145405 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf48d5d-7e0f-42b4-b446-166f96ba17e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:42:21.146056 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf48d5d-7e0f-42b4-b446-166f96ba17e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:42:21.147654 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf48d5d-7e0f-42b4-b446-166f96ba17e5 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:42:21.159455 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1201/2387] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Sep 6 04:42:21 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 04:42:21.163764 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9283f5b1-4e19-47c4-90d5-9e11d4dff51d admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:42:21.174238 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9283f5b1-4e19-47c4-90d5-9e11d4dff51d admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:42:21.178816 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e51cb7-0818-4446-8355-2f07e3bde3fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:42:21.179413 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e51cb7-0818-4446-8355-2f07e3bde3fc admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:42:21.179851 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e51cb7-0818-4446-8355-2f07e3bde3fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:42:21.180106 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e51cb7-0818-4446-8355-2f07e3bde3fc admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:42:21.180744 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e51cb7-0818-4446-8355-2f07e3bde3fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:42:21.182364 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e51cb7-0818-4446-8355-2f07e3bde3fc admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:42:21.193737 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1187/2388] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Sep 6 04:42:21 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 04:42:21.198036 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf48d5d-7e0f-42b4-b446-166f96ba17e5 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:42:21.208994 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf48d5d-7e0f-42b4-b446-166f96ba17e5 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:42:21.213833 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402fa4e0-100e-4348-bbc6-3e57ef01f95e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:42:21.214496 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402fa4e0-100e-4348-bbc6-3e57ef01f95e admin admin] REQUEST_METHOD: `PUT` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:42:21.214777 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402fa4e0-100e-4348-bbc6-3e57ef01f95e admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:42:21.215096 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402fa4e0-100e-4348-bbc6-3e57ef01f95e admin admin] PATH_INFO: `/v3/projects/3a1b663f43c74595a53237cec212992c/users/c1ee49c6f084470094ad9a55964bebcc/roles/29607b2de39f40d68bb6ceef64b8a732` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:42:21.231032 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402fa4e0-100e-4348-bbc6-3e57ef01f95e admin admin] RBAC: Authorizing `identity:create_grant(project_id=3a1b663f43c74595a53237cec212992c, user_id=c1ee49c6f084470094ad9a55964bebcc, role_id=29607b2de39f40d68bb6ceef64b8a732)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:42:21.233764 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402fa4e0-100e-4348-bbc6-3e57ef01f95e admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:42:21.322000 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1202/2389] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Sep 6 04:42:21 2022] PUT /identity/v3/projects/3a1b663f43c74595a53237cec212992c/users/c1ee49c6f084470094ad9a55964bebcc/roles/29607b2de39f40d68bb6ceef64b8a732 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 04:42:21.332852 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771539d3-7b65-4c11-9592-69cd677a9d2c None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:42:21.333226 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771539d3-7b65-4c11-9592-69cd677a9d2c None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:42:21.333848 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771539d3-7b65-4c11-9592-69cd677a9d2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:42:21.361946 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-771539d3-7b65-4c11-9592-69cd677a9d2c None None] MFA Rules not processed for user `c1ee49c6f084470094ad9a55964bebcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:42:21.363138 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-771539d3-7b65-4c11-9592-69cd677a9d2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:42:21.430495 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1188/2390] 10.222.0.30 () {58 vars in 958 bytes} [Tue Sep 6 04:42:21 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 04:42:21.448326 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402fa4e0-100e-4348-bbc6-3e57ef01f95e admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:42:21.448326 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-402fa4e0-100e-4348-bbc6-3e57ef01f95e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:42:21.463691 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:42:21.509683 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402fa4e0-100e-4348-bbc6-3e57ef01f95e admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:42:21.513520 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f587e3-ea5f-4374-b529-e7326e2d3b29 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd09b5ab675634a2eb4b03755743f7d0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b707733ce65c42edbc1fd95ad0a1c06c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:42:21.514146 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f587e3-ea5f-4374-b529-e7326e2d3b29 service cinder] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:42:21.514433 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f587e3-ea5f-4374-b529-e7326e2d3b29 service cinder] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:42:21.514712 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f587e3-ea5f-4374-b529-e7326e2d3b29 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:42:21.524431 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f587e3-ea5f-4374-b529-e7326e2d3b29 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:42:21.525951 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f587e3-ea5f-4374-b529-e7326e2d3b29 service cinder] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:42:21.537395 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1203/2391] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Sep 6 04:42:21 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 04:42:22.209977 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1059d2-f65a-433b-ac51-e53b36cadd43 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:42:22.210414 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1059d2-f65a-433b-ac51-e53b36cadd43 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:42:22.210577 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1059d2-f65a-433b-ac51-e53b36cadd43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:42:22.230382 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-9a1059d2-f65a-433b-ac51-e53b36cadd43 None None] MFA Rules not processed for user `5a0294ea40914bf290902307a787642d`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:42:22.231631 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-9a1059d2-f65a-433b-ac51-e53b36cadd43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:42:22.237883 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:42:22.291511 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1189/2392] 10.222.0.30 () {58 vars in 958 bytes} [Tue Sep 6 04:42:22 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 04:42:22.597923 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f587e3-ea5f-4374-b529-e7326e2d3b29 service cinder] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:42:22.599348 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-08f587e3-ea5f-4374-b529-e7326e2d3b29 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:42:22.656099 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f587e3-ea5f-4374-b529-e7326e2d3b29 service cinder] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:42:22.659853 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42d4974-3a66-496f-847c-50b968fa699e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90492440f6a6471e8392361c098428aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b707733ce65c42edbc1fd95ad0a1c06c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:42:22.660458 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42d4974-3a66-496f-847c-50b968fa699e service neutron] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:42:22.660741 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42d4974-3a66-496f-847c-50b968fa699e service neutron] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:42:22.661017 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42d4974-3a66-496f-847c-50b968fa699e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:42:22.662768 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-a42d4974-3a66-496f-847c-50b968fa699e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:42:22.672066 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42d4974-3a66-496f-847c-50b968fa699e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:42:22.674393 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42d4974-3a66-496f-847c-50b968fa699e service neutron] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:42:22.682319 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1204/2393] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Sep 6 04:42:22 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 04:42:23.085082 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1059d2-f65a-433b-ac51-e53b36cadd43 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:42:23.086549 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-9a1059d2-f65a-433b-ac51-e53b36cadd43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:42:23.164335 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1059d2-f65a-433b-ac51-e53b36cadd43 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:42:23.169317 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544474e9-bb91-458b-9562-731a38d72ef6 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d4f2e4dbd41441d84f03ddb30191e7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b707733ce65c42edbc1fd95ad0a1c06c', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:42:23.169932 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544474e9-bb91-458b-9562-731a38d72ef6 service placement] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:42:23.170209 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544474e9-bb91-458b-9562-731a38d72ef6 service placement] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:42:23.170478 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544474e9-bb91-458b-9562-731a38d72ef6 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:42:23.172358 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-544474e9-bb91-458b-9562-731a38d72ef6 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:42:23.183931 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544474e9-bb91-458b-9562-731a38d72ef6 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:42:23.185535 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544474e9-bb91-458b-9562-731a38d72ef6 service placement] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:42:23.195212 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1190/2394] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Sep 6 04:42:23 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 04:42:31.449114 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42d4974-3a66-496f-847c-50b968fa699e service neutron] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:42:31.571248 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42d4974-3a66-496f-847c-50b968fa699e service neutron] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:42:31.584089 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477d7045-c9b6-4aaf-a795-03a860615d1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:42:31.585161 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477d7045-c9b6-4aaf-a795-03a860615d1c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:42:31.586212 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477d7045-c9b6-4aaf-a795-03a860615d1c admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:42:31.587332 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477d7045-c9b6-4aaf-a795-03a860615d1c admin admin] PATH_INFO: `/v3/users/c1ee49c6f084470094ad9a55964bebcc` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:42:31.592892 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-477d7045-c9b6-4aaf-a795-03a860615d1c admin admin] RBAC: Authorizing `identity:delete_user(user_id=c1ee49c6f084470094ad9a55964bebcc)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:42:31.592892 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-477d7045-c9b6-4aaf-a795-03a860615d1c admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:42:31.833871 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-477d7045-c9b6-4aaf-a795-03a860615d1c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c1ee49c6f084470094ad9a55964bebcc'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:42:31.899056 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-477d7045-c9b6-4aaf-a795-03a860615d1c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c1ee49c6f084470094ad9a55964bebcc'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:42:31.900611 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-477d7045-c9b6-4aaf-a795-03a860615d1c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c1ee49c6f084470094ad9a55964bebcc'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:42:31.911419 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:42:31.927186 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-477d7045-c9b6-4aaf-a795-03a860615d1c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c1ee49c6f084470094ad9a55964bebcc'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:42:31.944465 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-477d7045-c9b6-4aaf-a795-03a860615d1c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c1ee49c6f084470094ad9a55964bebcc'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:42:31.955944 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1205/2395] 10.222.0.30 () {58 vars in 1224 bytes} [Tue Sep 6 04:42:31 2022] DELETE /identity/v3/users/c1ee49c6f084470094ad9a55964bebcc => generated 0 bytes in 507 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 04:42:31.961819 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f4ece7-b9d8-470f-81e5-c6727f254955 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:42:31.963698 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f4ece7-b9d8-470f-81e5-c6727f254955 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:42:31.964158 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f4ece7-b9d8-470f-81e5-c6727f254955 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:42:31.993451 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-07f4ece7-b9d8-470f-81e5-c6727f254955 None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:42:31.995060 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-07f4ece7-b9d8-470f-81e5-c6727f254955 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:42:32.086519 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1191/2396] 10.222.0.30 () {58 vars in 958 bytes} [Tue Sep 6 04:42:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 04:42:32.094617 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477d7045-c9b6-4aaf-a795-03a860615d1c admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:42:32.096836 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-477d7045-c9b6-4aaf-a795-03a860615d1c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:42:32.209779 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477d7045-c9b6-4aaf-a795-03a860615d1c admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:42:32.213873 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c19e122-9d12-4f8b-953d-7ef768f357ef service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90492440f6a6471e8392361c098428aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b707733ce65c42edbc1fd95ad0a1c06c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:42:32.214530 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c19e122-9d12-4f8b-953d-7ef768f357ef service neutron] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:42:32.214816 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c19e122-9d12-4f8b-953d-7ef768f357ef service neutron] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:42:32.215092 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c19e122-9d12-4f8b-953d-7ef768f357ef service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:42:32.227274 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c19e122-9d12-4f8b-953d-7ef768f357ef service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:42:32.229120 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c19e122-9d12-4f8b-953d-7ef768f357ef service neutron] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:42:32.240803 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1206/2397] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Sep 6 04:42:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 04:42:32.689092 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f4ece7-b9d8-470f-81e5-c6727f254955 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:42:32.691303 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-07f4ece7-b9d8-470f-81e5-c6727f254955 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:42:32.703855 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f4ece7-b9d8-470f-81e5-c6727f254955 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:42:32.709001 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aaf3429-fa5f-4b59-a0f2-4db18a44d6a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:42:32.709691 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aaf3429-fa5f-4b59-a0f2-4db18a44d6a1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:42:32.709975 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aaf3429-fa5f-4b59-a0f2-4db18a44d6a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:42:32.710251 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aaf3429-fa5f-4b59-a0f2-4db18a44d6a1 admin admin] PATH_INFO: `/v3/projects/3a1b663f43c74595a53237cec212992c` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:42:32.711977 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aaf3429-fa5f-4b59-a0f2-4db18a44d6a1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3a1b663f43c74595a53237cec212992c)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:42:32.713621 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aaf3429-fa5f-4b59-a0f2-4db18a44d6a1 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:42:32.870061 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:42:32.875838 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-8aaf3429-fa5f-4b59-a0f2-4db18a44d6a1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3a1b663f43c74595a53237cec212992c', 'request_id': 'req-8aaf3429-fa5f-4b59-a0f2-4db18a44d6a1'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:42:32.990170 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-8aaf3429-fa5f-4b59-a0f2-4db18a44d6a1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3a1b663f43c74595a53237cec212992c', 'request_id': 'req-8aaf3429-fa5f-4b59-a0f2-4db18a44d6a1'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:42:32.998383 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-8aaf3429-fa5f-4b59-a0f2-4db18a44d6a1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3a1b663f43c74595a53237cec212992c', 'request_id': 'req-8aaf3429-fa5f-4b59-a0f2-4db18a44d6a1'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:42:33.013146 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-8aaf3429-fa5f-4b59-a0f2-4db18a44d6a1 admin admin] The token cache is being invalidate because project 3a1b663f43c74595a53237cec212992c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81697) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 04:42:33.013627 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-8aaf3429-fa5f-4b59-a0f2-4db18a44d6a1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:42:33.014301 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-8aaf3429-fa5f-4b59-a0f2-4db18a44d6a1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:42:33.016000 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1192/2398] 10.222.0.30 () {58 vars in 1233 bytes} [Tue Sep 6 04:42:32 2022] DELETE /identity/v3/projects/3a1b663f43c74595a53237cec212992c => generated 0 bytes in 327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 04:42:33.021780 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbc3212-6ca2-473a-8082-4d2bac1c15c3 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:42:33.029108 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbc3212-6ca2-473a-8082-4d2bac1c15c3 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:42:33.029108 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbc3212-6ca2-473a-8082-4d2bac1c15c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:42:33.044845 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-bcbc3212-6ca2-473a-8082-4d2bac1c15c3 None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:42:33.046118 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-bcbc3212-6ca2-473a-8082-4d2bac1c15c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:42:33.110565 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1207/2399] 10.222.0.30 () {58 vars in 958 bytes} [Tue Sep 6 04:42:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 04:42:33.121686 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cfbbef-40cf-479d-bab5-dccd03cc049b None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:42:33.122060 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cfbbef-40cf-479d-bab5-dccd03cc049b None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:42:33.122407 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cfbbef-40cf-479d-bab5-dccd03cc049b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:42:33.144027 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-a2cfbbef-40cf-479d-bab5-dccd03cc049b None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:42:33.145989 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-a2cfbbef-40cf-479d-bab5-dccd03cc049b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:42:33.160838 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1193/2400] 10.222.0.30 () {58 vars in 958 bytes} [Tue Sep 6 04:42:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 04:42:33.161965 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbc3212-6ca2-473a-8082-4d2bac1c15c3 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:42:33.172438 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbc3212-6ca2-473a-8082-4d2bac1c15c3 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:42:33.187021 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0796c469-40d1-40f6-9118-394bec737792 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:42:33.188619 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0796c469-40d1-40f6-9118-394bec737792 admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:42:33.188619 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0796c469-40d1-40f6-9118-394bec737792 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:42:33.188619 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0796c469-40d1-40f6-9118-394bec737792 admin admin] PATH_INFO: `/v3/domains` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:42:33.188941 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0796c469-40d1-40f6-9118-394bec737792 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:42:33.189254 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0796c469-40d1-40f6-9118-394bec737792 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:42:33.191601 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0796c469-40d1-40f6-9118-394bec737792 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:42:33.206172 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1208/2401] 10.222.0.30 () {58 vars in 1153 bytes} [Tue Sep 6 04:42:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 04:42:33.210415 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2cfbbef-40cf-479d-bab5-dccd03cc049b None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:42:33.225836 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2cfbbef-40cf-479d-bab5-dccd03cc049b None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:42:33.239367 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843a4404-4ec5-499d-90fa-d0b3665d4d35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:42:33.239367 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843a4404-4ec5-499d-90fa-d0b3665d4d35 admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:42:33.239367 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843a4404-4ec5-499d-90fa-d0b3665d4d35 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:42:33.239367 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843a4404-4ec5-499d-90fa-d0b3665d4d35 admin admin] PATH_INFO: `/v3/projects` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:42:33.240191 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-843a4404-4ec5-499d-90fa-d0b3665d4d35 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:42:33.240191 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-843a4404-4ec5-499d-90fa-d0b3665d4d35 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:42:33.312312 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1194/2402] 10.222.0.30 () {60 vars in 1153 bytes} [Tue Sep 6 04:42:33 2022] POST /identity/v3/projects => generated 385 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 04:42:33.319131 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0796c469-40d1-40f6-9118-394bec737792 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:42:33.335021 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0796c469-40d1-40f6-9118-394bec737792 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:42:33.417995 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a7bbec-1b39-4358-a9ac-dc94abd6d93e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:42:33.418747 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a7bbec-1b39-4358-a9ac-dc94abd6d93e admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:42:33.419062 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a7bbec-1b39-4358-a9ac-dc94abd6d93e admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:42:33.419517 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a7bbec-1b39-4358-a9ac-dc94abd6d93e admin admin] PATH_INFO: `/v3/users` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:42:33.420155 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a7bbec-1b39-4358-a9ac-dc94abd6d93e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:42:33.422300 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a7bbec-1b39-4358-a9ac-dc94abd6d93e admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:42:33.515790 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:42:33.522205 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1209/2403] 10.222.0.30 () {60 vars in 1144 bytes} [Tue Sep 6 04:42:33 2022] POST /identity/v3/users => generated 293 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 04:42:33.526368 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843a4404-4ec5-499d-90fa-d0b3665d4d35 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:42:33.537196 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843a4404-4ec5-499d-90fa-d0b3665d4d35 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:42:33.541925 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec8d2ad-15d4-4f52-9bcb-ca53c9099f30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:42:33.542607 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec8d2ad-15d4-4f52-9bcb-ca53c9099f30 admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:42:33.542811 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec8d2ad-15d4-4f52-9bcb-ca53c9099f30 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:42:33.543079 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec8d2ad-15d4-4f52-9bcb-ca53c9099f30 admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:42:33.543717 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec8d2ad-15d4-4f52-9bcb-ca53c9099f30 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:42:33.545297 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec8d2ad-15d4-4f52-9bcb-ca53c9099f30 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:42:33.555873 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1195/2404] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Sep 6 04:42:33 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 04:42:33.559888 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a7bbec-1b39-4358-a9ac-dc94abd6d93e admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:42:33.569948 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a7bbec-1b39-4358-a9ac-dc94abd6d93e admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:42:33.574608 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1878e457-7ea6-468b-8988-a92578eae2f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:42:33.575210 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1878e457-7ea6-468b-8988-a92578eae2f3 admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:42:33.575492 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1878e457-7ea6-468b-8988-a92578eae2f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:42:33.575766 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1878e457-7ea6-468b-8988-a92578eae2f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:42:33.576409 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1878e457-7ea6-468b-8988-a92578eae2f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:42:33.578099 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1878e457-7ea6-468b-8988-a92578eae2f3 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:42:33.589373 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1210/2405] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Sep 6 04:42:33 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 04:42:33.593397 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec8d2ad-15d4-4f52-9bcb-ca53c9099f30 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:42:33.604028 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec8d2ad-15d4-4f52-9bcb-ca53c9099f30 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:42:33.608627 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e7e5a3-9fe8-4cba-a112-60fbbfd38b20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:42:33.609281 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e7e5a3-9fe8-4cba-a112-60fbbfd38b20 admin admin] REQUEST_METHOD: `PUT` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:42:33.609593 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e7e5a3-9fe8-4cba-a112-60fbbfd38b20 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:42:33.609870 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e7e5a3-9fe8-4cba-a112-60fbbfd38b20 admin admin] PATH_INFO: `/v3/projects/623b9f3be0f24e2ea1caea0798b30704/users/9f07caa54c8f47778bb984bfd9800dc0/roles/29607b2de39f40d68bb6ceef64b8a732` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:42:33.625191 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e7e5a3-9fe8-4cba-a112-60fbbfd38b20 admin admin] RBAC: Authorizing `identity:create_grant(project_id=623b9f3be0f24e2ea1caea0798b30704, user_id=9f07caa54c8f47778bb984bfd9800dc0, role_id=29607b2de39f40d68bb6ceef64b8a732)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:42:33.628174 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e7e5a3-9fe8-4cba-a112-60fbbfd38b20 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:42:33.758858 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1196/2406] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Sep 6 04:42:33 2022] PUT /identity/v3/projects/623b9f3be0f24e2ea1caea0798b30704/users/9f07caa54c8f47778bb984bfd9800dc0/roles/29607b2de39f40d68bb6ceef64b8a732 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 04:42:33.763913 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88883594-75d9-4ddf-bdae-c7b371581894 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:42:33.764206 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88883594-75d9-4ddf-bdae-c7b371581894 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:42:33.764484 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88883594-75d9-4ddf-bdae-c7b371581894 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:42:33.793007 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-88883594-75d9-4ddf-bdae-c7b371581894 None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:42:33.794356 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-88883594-75d9-4ddf-bdae-c7b371581894 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:42:33.861725 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1211/2407] 10.222.0.30 () {58 vars in 958 bytes} [Tue Sep 6 04:42:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 04:42:33.871610 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e7e5a3-9fe8-4cba-a112-60fbbfd38b20 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:42:33.873520 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-93e7e5a3-9fe8-4cba-a112-60fbbfd38b20 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:42:33.949711 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e7e5a3-9fe8-4cba-a112-60fbbfd38b20 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:42:33.954774 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e928e3df-ebdf-4e3a-9862-7bc9cbd64283 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90492440f6a6471e8392361c098428aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b707733ce65c42edbc1fd95ad0a1c06c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:42:33.955768 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e928e3df-ebdf-4e3a-9862-7bc9cbd64283 service neutron] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:42:33.956238 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e928e3df-ebdf-4e3a-9862-7bc9cbd64283 service neutron] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:42:33.956680 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e928e3df-ebdf-4e3a-9862-7bc9cbd64283 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:42:33.966938 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e928e3df-ebdf-4e3a-9862-7bc9cbd64283 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:42:33.968396 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e928e3df-ebdf-4e3a-9862-7bc9cbd64283 service neutron] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:42:33.981288 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1197/2408] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Sep 6 04:42:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 04:42:34.977743 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057c0d72-487b-4762-8626-a45e587663de None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:42:34.978227 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057c0d72-487b-4762-8626-a45e587663de None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:42:34.979113 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057c0d72-487b-4762-8626-a45e587663de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:42:34.998539 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-057c0d72-487b-4762-8626-a45e587663de None None] MFA Rules not processed for user `4b6c761af9c5457ea1e481ce400aded8`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:42:34.999882 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-057c0d72-487b-4762-8626-a45e587663de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:42:35.062125 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1212/2409] 10.222.0.30 () {58 vars in 958 bytes} [Tue Sep 6 04:42:34 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 04:42:40.191190 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e05efa9-e55e-483b-8862-eb89006a4da3 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:42:40.191684 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e05efa9-e55e-483b-8862-eb89006a4da3 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:42:40.191770 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e05efa9-e55e-483b-8862-eb89006a4da3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:42:40.229684 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-2e05efa9-e55e-483b-8862-eb89006a4da3 None None] MFA Rules not processed for user `9f07caa54c8f47778bb984bfd9800dc0`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:42:40.230917 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-2e05efa9-e55e-483b-8862-eb89006a4da3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:42:40.315154 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=93,keystone:DELETE=4,keystone:INSERT=3 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:42:40.320348 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1198/2410] 10.222.0.30 () {58 vars in 958 bytes} [Tue Sep 6 04:42:40 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 04:42:40.321619 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057c0d72-487b-4762-8626-a45e587663de None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:42:40.335255 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057c0d72-487b-4762-8626-a45e587663de None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:42:40.340597 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c3dc0d-0a81-4895-8bd5-282859154edd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:42:40.343861 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c3dc0d-0a81-4895-8bd5-282859154edd admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:42:40.343861 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c3dc0d-0a81-4895-8bd5-282859154edd admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:42:40.343861 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c3dc0d-0a81-4895-8bd5-282859154edd admin admin] PATH_INFO: `/v3/projects` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:42:40.343861 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c3dc0d-0a81-4895-8bd5-282859154edd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:42:40.344649 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c3dc0d-0a81-4895-8bd5-282859154edd admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:42:40.466691 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1213/2411] 10.222.0.30 () {60 vars in 1153 bytes} [Tue Sep 6 04:42:40 2022] POST /identity/v3/projects => generated 383 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 04:42:40.472933 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e05efa9-e55e-483b-8862-eb89006a4da3 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:42:40.486471 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e05efa9-e55e-483b-8862-eb89006a4da3 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:42:40.548880 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c648c51-23d8-4e6a-a2a3-52f0ce57e165 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:42:40.549788 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c648c51-23d8-4e6a-a2a3-52f0ce57e165 admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:42:40.550068 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c648c51-23d8-4e6a-a2a3-52f0ce57e165 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:42:40.550344 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c648c51-23d8-4e6a-a2a3-52f0ce57e165 admin admin] PATH_INFO: `/v3/users` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:42:40.550982 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c648c51-23d8-4e6a-a2a3-52f0ce57e165 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:42:40.553033 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c648c51-23d8-4e6a-a2a3-52f0ce57e165 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:42:40.676955 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1199/2412] 10.222.0.30 () {60 vars in 1144 bytes} [Tue Sep 6 04:42:40 2022] POST /identity/v3/users => generated 298 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 04:42:40.681061 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c3dc0d-0a81-4895-8bd5-282859154edd admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:42:40.693599 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c3dc0d-0a81-4895-8bd5-282859154edd admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:42:40.700737 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11905c67-e085-40a7-ac5d-7280e8bee7ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:42:40.700918 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11905c67-e085-40a7-ac5d-7280e8bee7ab admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:42:40.701440 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11905c67-e085-40a7-ac5d-7280e8bee7ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:42:40.701720 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11905c67-e085-40a7-ac5d-7280e8bee7ab admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:42:40.702502 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11905c67-e085-40a7-ac5d-7280e8bee7ab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:42:40.706681 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11905c67-e085-40a7-ac5d-7280e8bee7ab admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:42:40.719771 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1214/2413] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Sep 6 04:42:40 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 04:42:40.724714 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c648c51-23d8-4e6a-a2a3-52f0ce57e165 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:42:40.736016 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c648c51-23d8-4e6a-a2a3-52f0ce57e165 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:42:40.743585 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65365b0-cf9d-45e4-a255-466bc606cac6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:42:40.744545 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65365b0-cf9d-45e4-a255-466bc606cac6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:42:40.755895 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65365b0-cf9d-45e4-a255-466bc606cac6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:42:40.755895 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65365b0-cf9d-45e4-a255-466bc606cac6 admin admin] PATH_INFO: `/v3/projects/e5f9492622a543b4808f5816cb2afdf9/users/dae02250da564a23bc090baffe73a30f/roles/ea4c814713b149108e488de855b025e4` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:42:40.764695 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65365b0-cf9d-45e4-a255-466bc606cac6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e5f9492622a543b4808f5816cb2afdf9, user_id=dae02250da564a23bc090baffe73a30f, role_id=ea4c814713b149108e488de855b025e4)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:42:40.767904 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65365b0-cf9d-45e4-a255-466bc606cac6 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:42:40.860765 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1200/2414] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Sep 6 04:42:40 2022] PUT /identity/v3/projects/e5f9492622a543b4808f5816cb2afdf9/users/dae02250da564a23bc090baffe73a30f/roles/ea4c814713b149108e488de855b025e4 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 04:42:40.865723 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11905c67-e085-40a7-ac5d-7280e8bee7ab admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:42:40.880588 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11905c67-e085-40a7-ac5d-7280e8bee7ab admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:42:40.957203 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a2e945-1f5b-4026-8f2d-418f5f0005f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:42:40.957963 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a2e945-1f5b-4026-8f2d-418f5f0005f2 admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:42:40.958244 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a2e945-1f5b-4026-8f2d-418f5f0005f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:42:40.958689 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a2e945-1f5b-4026-8f2d-418f5f0005f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:42:40.959527 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a2e945-1f5b-4026-8f2d-418f5f0005f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:42:40.961280 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a2e945-1f5b-4026-8f2d-418f5f0005f2 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:42:40.963269 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:42:40.975358 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1215/2415] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Sep 6 04:42:40 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 04:42:40.979288 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65365b0-cf9d-45e4-a255-466bc606cac6 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:42:40.992419 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65365b0-cf9d-45e4-a255-466bc606cac6 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:42:41.010038 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2680aff7-764f-419e-a4e4-67b652d53312 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:42:41.010724 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2680aff7-764f-419e-a4e4-67b652d53312 admin admin] REQUEST_METHOD: `PUT` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:42:41.011026 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2680aff7-764f-419e-a4e4-67b652d53312 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:42:41.011267 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2680aff7-764f-419e-a4e4-67b652d53312 admin admin] PATH_INFO: `/v3/projects/e5f9492622a543b4808f5816cb2afdf9/users/dae02250da564a23bc090baffe73a30f/roles/ea4c814713b149108e488de855b025e4` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:42:41.015411 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2680aff7-764f-419e-a4e4-67b652d53312 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e5f9492622a543b4808f5816cb2afdf9, user_id=dae02250da564a23bc090baffe73a30f, role_id=ea4c814713b149108e488de855b025e4)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:42:41.018496 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2680aff7-764f-419e-a4e4-67b652d53312 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:42:41.047365 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1201/2416] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Sep 6 04:42:40 2022] PUT /identity/v3/projects/e5f9492622a543b4808f5816cb2afdf9/users/dae02250da564a23bc090baffe73a30f/roles/ea4c814713b149108e488de855b025e4 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 04:42:41.414414 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a2e945-1f5b-4026-8f2d-418f5f0005f2 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:42:41.416062 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-75a2e945-1f5b-4026-8f2d-418f5f0005f2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:42:41.519382 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a2e945-1f5b-4026-8f2d-418f5f0005f2 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:42:41.606968 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ad1da7-1330-4cc6-9317-b01e61f3e4c5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9db138b77afa4b068244b8ef715404fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b707733ce65c42edbc1fd95ad0a1c06c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:42:41.607739 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ad1da7-1330-4cc6-9317-b01e61f3e4c5 service nova] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:42:41.608091 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ad1da7-1330-4cc6-9317-b01e61f3e4c5 service nova] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:42:41.608431 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ad1da7-1330-4cc6-9317-b01e61f3e4c5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:42:41.610598 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-36ad1da7-1330-4cc6-9317-b01e61f3e4c5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:42:41.625539 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ad1da7-1330-4cc6-9317-b01e61f3e4c5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:42:41.626195 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ad1da7-1330-4cc6-9317-b01e61f3e4c5 service nova] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:42:41.649544 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1216/2417] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Sep 6 04:42:41 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 236 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 04:42:49.658379 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6801df14-6c01-43a3-88c8-ff598d5324e8 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:42:49.658901 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6801df14-6c01-43a3-88c8-ff598d5324e8 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:42:49.658901 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6801df14-6c01-43a3-88c8-ff598d5324e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:42:49.702684 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-6801df14-6c01-43a3-88c8-ff598d5324e8 None None] MFA Rules not processed for user `dae02250da564a23bc090baffe73a30f`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:42:49.708732 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-6801df14-6c01-43a3-88c8-ff598d5324e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:42:49.828614 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1202/2418] 10.222.0.30 () {58 vars in 958 bytes} [Tue Sep 6 04:42:49 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 04:42:49.853919 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ad1da7-1330-4cc6-9317-b01e61f3e4c5 service nova] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:42:49.855396 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-36ad1da7-1330-4cc6-9317-b01e61f3e4c5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:42:49.919228 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ad1da7-1330-4cc6-9317-b01e61f3e4c5 service nova] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:42:49.922999 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50671a97-521f-4c63-9a8a-2527c7ad87a7 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae5c31fb1d0342c0937bbc83a415113d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b707733ce65c42edbc1fd95ad0a1c06c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:42:49.923665 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50671a97-521f-4c63-9a8a-2527c7ad87a7 service glance] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:42:49.924448 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50671a97-521f-4c63-9a8a-2527c7ad87a7 service glance] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:42:49.924722 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50671a97-521f-4c63-9a8a-2527c7ad87a7 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:42:49.961096 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:42:49.988553 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50671a97-521f-4c63-9a8a-2527c7ad87a7 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:42:49.990132 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50671a97-521f-4c63-9a8a-2527c7ad87a7 service glance] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:42:49.999572 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1217/2419] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Sep 6 04:42:49 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 04:42:50.274628 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e205f692-8f61-4a3b-a6f2-f2be3fbe68af None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:42:50.275007 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e205f692-8f61-4a3b-a6f2-f2be3fbe68af None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:42:50.275361 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e205f692-8f61-4a3b-a6f2-f2be3fbe68af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:42:50.297009 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-e205f692-8f61-4a3b-a6f2-f2be3fbe68af None None] MFA Rules not processed for user `da5fed17101a4935bcc38b72caf9d263`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:42:50.298492 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-e205f692-8f61-4a3b-a6f2-f2be3fbe68af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:42:50.342171 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:42:50.379064 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1203/2420] 10.222.0.30 () {58 vars in 1009 bytes} [Tue Sep 6 04:42:50 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 04:42:50.531022 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50671a97-521f-4c63-9a8a-2527c7ad87a7 service glance] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:42:50.532460 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-50671a97-521f-4c63-9a8a-2527c7ad87a7 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:42:50.616944 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50671a97-521f-4c63-9a8a-2527c7ad87a7 service glance] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:42:50.622744 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176aef70-fdd6-41ff-9f03-d7aff3c5b710 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a676ec7f92a144edbeca8e6029ba9ee6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b707733ce65c42edbc1fd95ad0a1c06c', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:42:50.623375 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176aef70-fdd6-41ff-9f03-d7aff3c5b710 service swift] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:42:50.623689 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176aef70-fdd6-41ff-9f03-d7aff3c5b710 service swift] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:42:50.623969 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176aef70-fdd6-41ff-9f03-d7aff3c5b710 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:42:50.635886 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176aef70-fdd6-41ff-9f03-d7aff3c5b710 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:42:50.637681 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176aef70-fdd6-41ff-9f03-d7aff3c5b710 service swift] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:42:50.649258 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1218/2421] 10.222.0.30 () {60 vars in 1384 bytes} [Tue Sep 6 04:42:50 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 06 04:42:52.030219 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e205f692-8f61-4a3b-a6f2-f2be3fbe68af None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:42:52.053469 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e205f692-8f61-4a3b-a6f2-f2be3fbe68af None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:42:52.063997 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b6fa61-b005-43d5-8f57-e9fa4d75fb96 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9db138b77afa4b068244b8ef715404fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b707733ce65c42edbc1fd95ad0a1c06c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:42:52.065416 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b6fa61-b005-43d5-8f57-e9fa4d75fb96 service nova] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:42:52.068940 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b6fa61-b005-43d5-8f57-e9fa4d75fb96 service nova] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:42:52.068940 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b6fa61-b005-43d5-8f57-e9fa4d75fb96 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:42:52.068940 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-59b6fa61-b005-43d5-8f57-e9fa4d75fb96 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:42:52.161337 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b6fa61-b005-43d5-8f57-e9fa4d75fb96 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:42:52.164930 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b6fa61-b005-43d5-8f57-e9fa4d75fb96 service nova] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:42:52.178236 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1204/2422] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Sep 6 04:42:52 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 04:42:52.714873 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32af9b3a-4052-430a-9190-1170828f4ad8 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:42:52.715344 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32af9b3a-4052-430a-9190-1170828f4ad8 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:42:52.715507 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32af9b3a-4052-430a-9190-1170828f4ad8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:42:52.736274 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-32af9b3a-4052-430a-9190-1170828f4ad8 None None] MFA Rules not processed for user `1ca147cfc97149dd87e533123a262cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:42:52.737671 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-32af9b3a-4052-430a-9190-1170828f4ad8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:42:52.831227 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1219/2423] 10.222.0.30 () {58 vars in 958 bytes} [Tue Sep 6 04:42:52 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 04:42:53.199083 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b6fa61-b005-43d5-8f57-e9fa4d75fb96 service nova] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:42:53.200820 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-59b6fa61-b005-43d5-8f57-e9fa4d75fb96 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:42:53.292253 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b6fa61-b005-43d5-8f57-e9fa4d75fb96 service nova] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:42:53.296357 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b415c90b-a569-4752-8e04-1cd374d16d6e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90492440f6a6471e8392361c098428aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b707733ce65c42edbc1fd95ad0a1c06c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:42:53.297285 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b415c90b-a569-4752-8e04-1cd374d16d6e service neutron] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:42:53.297822 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b415c90b-a569-4752-8e04-1cd374d16d6e service neutron] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:42:53.298414 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b415c90b-a569-4752-8e04-1cd374d16d6e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:42:53.302923 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-b415c90b-a569-4752-8e04-1cd374d16d6e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:42:53.332592 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b415c90b-a569-4752-8e04-1cd374d16d6e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:42:53.338568 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b415c90b-a569-4752-8e04-1cd374d16d6e service neutron] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:42:53.357153 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1205/2424] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Sep 6 04:42:53 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 04:43:02.317476 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7baa0ba8-d86c-42d3-a5e3-4fad3462ced4 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:43:02.317476 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7baa0ba8-d86c-42d3-a5e3-4fad3462ced4 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:43:02.317953 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7baa0ba8-d86c-42d3-a5e3-4fad3462ced4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:43:02.337763 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-7baa0ba8-d86c-42d3-a5e3-4fad3462ced4 None None] MFA Rules not processed for user `9f07caa54c8f47778bb984bfd9800dc0`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:43:02.338992 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-7baa0ba8-d86c-42d3-a5e3-4fad3462ced4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:43:02.347101 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1220/2425] 10.222.0.30 () {58 vars in 958 bytes} [Tue Sep 6 04:43:02 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 04:43:03.345591 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=63 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:43:09.093333 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0e00db-c79b-4aec-a028-5dbbb4f9f5b3 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:43:09.094338 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0e00db-c79b-4aec-a028-5dbbb4f9f5b3 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:43:09.094626 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0e00db-c79b-4aec-a028-5dbbb4f9f5b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:43:09.125528 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-bd0e00db-c79b-4aec-a028-5dbbb4f9f5b3 None None] MFA Rules not processed for user `da5fed17101a4935bcc38b72caf9d263`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:43:09.128993 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-bd0e00db-c79b-4aec-a028-5dbbb4f9f5b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:43:09.152368 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7baa0ba8-d86c-42d3-a5e3-4fad3462ced4 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:43:09.161513 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1206/2426] 10.222.0.30 () {58 vars in 1009 bytes} [Tue Sep 6 04:43:09 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 68 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 04:43:09.169844 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7baa0ba8-d86c-42d3-a5e3-4fad3462ced4 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:43:09.176172 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a61f39-c2a3-4c6c-a405-21c83f25d333 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a676ec7f92a144edbeca8e6029ba9ee6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b707733ce65c42edbc1fd95ad0a1c06c', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:43:09.178647 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a61f39-c2a3-4c6c-a405-21c83f25d333 service swift] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:43:09.178647 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a61f39-c2a3-4c6c-a405-21c83f25d333 service swift] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:43:09.178647 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a61f39-c2a3-4c6c-a405-21c83f25d333 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:43:09.188169 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a61f39-c2a3-4c6c-a405-21c83f25d333 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:43:09.190508 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a61f39-c2a3-4c6c-a405-21c83f25d333 service swift] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:43:09.200327 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1221/2427] 10.222.0.30 () {60 vars in 1384 bytes} [Tue Sep 6 04:43:09 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 06 04:43:19.114528 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=3 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:43:19.196976 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=78 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:43:34.269912 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0e00db-c79b-4aec-a028-5dbbb4f9f5b3 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:43:34.291445 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0e00db-c79b-4aec-a028-5dbbb4f9f5b3 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:43:34.296022 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347d73ea-17d6-4200-a44b-7ec448215d30 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90492440f6a6471e8392361c098428aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b707733ce65c42edbc1fd95ad0a1c06c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:43:34.296645 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347d73ea-17d6-4200-a44b-7ec448215d30 service neutron] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:43:34.297191 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347d73ea-17d6-4200-a44b-7ec448215d30 service neutron] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:43:34.297501 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347d73ea-17d6-4200-a44b-7ec448215d30 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:43:34.299364 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-347d73ea-17d6-4200-a44b-7ec448215d30 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:43:34.429900 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347d73ea-17d6-4200-a44b-7ec448215d30 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:43:34.431366 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347d73ea-17d6-4200-a44b-7ec448215d30 service neutron] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:43:34.451356 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1207/2428] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Sep 6 04:43:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 06 04:43:38.751681 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a61f39-c2a3-4c6c-a405-21c83f25d333 service swift] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:43:38.753626 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-b2a61f39-c2a3-4c6c-a405-21c83f25d333 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:43:38.792656 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a61f39-c2a3-4c6c-a405-21c83f25d333 service swift] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:43:38.797431 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344bb8f7-6589-47e4-b656-540b5a42bb4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:43:38.798057 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344bb8f7-6589-47e4-b656-540b5a42bb4b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:43:38.798336 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344bb8f7-6589-47e4-b656-540b5a42bb4b admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:43:38.798630 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344bb8f7-6589-47e4-b656-540b5a42bb4b admin admin] PATH_INFO: `/v3/users/dec1c17550b849dbadbc16e98530b455` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:43:38.799761 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344bb8f7-6589-47e4-b656-540b5a42bb4b admin admin] RBAC: Authorizing `identity:delete_user(user_id=dec1c17550b849dbadbc16e98530b455)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:43:38.801721 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344bb8f7-6589-47e4-b656-540b5a42bb4b admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:43:39.060013 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-344bb8f7-6589-47e4-b656-540b5a42bb4b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dec1c17550b849dbadbc16e98530b455'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:43:39.154906 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-344bb8f7-6589-47e4-b656-540b5a42bb4b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dec1c17550b849dbadbc16e98530b455'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:43:39.155472 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-344bb8f7-6589-47e4-b656-540b5a42bb4b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dec1c17550b849dbadbc16e98530b455'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:43:39.170072 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-344bb8f7-6589-47e4-b656-540b5a42bb4b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dec1c17550b849dbadbc16e98530b455'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:43:39.177817 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-344bb8f7-6589-47e4-b656-540b5a42bb4b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dec1c17550b849dbadbc16e98530b455'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:43:39.184163 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1222/2429] 10.222.0.30 () {58 vars in 1224 bytes} [Tue Sep 6 04:43:38 2022] DELETE /identity/v3/users/dec1c17550b849dbadbc16e98530b455 => generated 0 bytes in 433 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 04:43:39.436142 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347d73ea-17d6-4200-a44b-7ec448215d30 service neutron] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:43:39.447717 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347d73ea-17d6-4200-a44b-7ec448215d30 service neutron] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:43:39.509757 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbbc927b-c3bf-41d8-bca0-4dd4263f1b77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:43:39.510394 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbc927b-c3bf-41d8-bca0-4dd4263f1b77 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:43:39.510678 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbc927b-c3bf-41d8-bca0-4dd4263f1b77 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:43:39.510956 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbc927b-c3bf-41d8-bca0-4dd4263f1b77 admin admin] PATH_INFO: `/v3/projects/7aa4ca550edd4876a4c6c56deedf9d70` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:43:39.512081 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbbc927b-c3bf-41d8-bca0-4dd4263f1b77 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7aa4ca550edd4876a4c6c56deedf9d70)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:43:39.514144 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbbc927b-c3bf-41d8-bca0-4dd4263f1b77 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:43:39.615169 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:43:39.631225 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-fbbc927b-c3bf-41d8-bca0-4dd4263f1b77 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7aa4ca550edd4876a4c6c56deedf9d70', 'request_id': 'req-fbbc927b-c3bf-41d8-bca0-4dd4263f1b77'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:43:39.733289 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-fbbc927b-c3bf-41d8-bca0-4dd4263f1b77 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7aa4ca550edd4876a4c6c56deedf9d70', 'request_id': 'req-fbbc927b-c3bf-41d8-bca0-4dd4263f1b77'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:43:39.740679 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-fbbc927b-c3bf-41d8-bca0-4dd4263f1b77 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7aa4ca550edd4876a4c6c56deedf9d70', 'request_id': 'req-fbbc927b-c3bf-41d8-bca0-4dd4263f1b77'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:43:39.751008 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-fbbc927b-c3bf-41d8-bca0-4dd4263f1b77 admin admin] The token cache is being invalidate because project 7aa4ca550edd4876a4c6c56deedf9d70 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81697) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 04:43:39.751293 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-fbbc927b-c3bf-41d8-bca0-4dd4263f1b77 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:43:39.751921 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-fbbc927b-c3bf-41d8-bca0-4dd4263f1b77 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:43:39.754097 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1208/2430] 10.222.0.30 () {58 vars in 1233 bytes} [Tue Sep 6 04:43:39 2022] DELETE /identity/v3/projects/7aa4ca550edd4876a4c6c56deedf9d70 => generated 0 bytes in 319 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 04:43:39.759166 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0341efc8-c421-4235-bb4b-7bd6faaef992 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:43:39.759645 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0341efc8-c421-4235-bb4b-7bd6faaef992 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:43:39.759742 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0341efc8-c421-4235-bb4b-7bd6faaef992 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:43:39.820508 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-0341efc8-c421-4235-bb4b-7bd6faaef992 None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:43:39.821774 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-0341efc8-c421-4235-bb4b-7bd6faaef992 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:43:39.883264 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1223/2431] 10.222.0.30 () {58 vars in 958 bytes} [Tue Sep 6 04:43:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 04:43:39.890903 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1a3cda-a544-4c58-8cc5-9ba848ef84ba None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:43:39.891191 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1a3cda-a544-4c58-8cc5-9ba848ef84ba None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:43:39.891690 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1a3cda-a544-4c58-8cc5-9ba848ef84ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:43:39.914081 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-7d1a3cda-a544-4c58-8cc5-9ba848ef84ba None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:43:39.915385 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-7d1a3cda-a544-4c58-8cc5-9ba848ef84ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:43:39.923750 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1209/2432] 10.222.0.30 () {58 vars in 958 bytes} [Tue Sep 6 04:43:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 04:43:39.928719 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0341efc8-c421-4235-bb4b-7bd6faaef992 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:43:39.939352 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0341efc8-c421-4235-bb4b-7bd6faaef992 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:43:39.944572 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6113e934-fc90-41ee-b710-8e4c7bdc17be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:43:39.946863 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6113e934-fc90-41ee-b710-8e4c7bdc17be admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:43:39.946863 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6113e934-fc90-41ee-b710-8e4c7bdc17be admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:43:39.946863 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6113e934-fc90-41ee-b710-8e4c7bdc17be admin admin] PATH_INFO: `/v3/domains` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:43:39.946863 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6113e934-fc90-41ee-b710-8e4c7bdc17be admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:43:39.946863 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6113e934-fc90-41ee-b710-8e4c7bdc17be admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:43:39.948554 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6113e934-fc90-41ee-b710-8e4c7bdc17be admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:43:39.960829 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1224/2433] 10.222.0.30 () {58 vars in 1153 bytes} [Tue Sep 6 04:43:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:43:39.965230 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1a3cda-a544-4c58-8cc5-9ba848ef84ba None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:43:39.974993 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1a3cda-a544-4c58-8cc5-9ba848ef84ba None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:43:39.979773 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83e8b12-8806-42cd-83b5-70585af71980 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:43:39.980371 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83e8b12-8806-42cd-83b5-70585af71980 admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:43:39.980649 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83e8b12-8806-42cd-83b5-70585af71980 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:43:39.980920 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83e8b12-8806-42cd-83b5-70585af71980 admin admin] PATH_INFO: `/v3/projects` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:43:39.981628 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a83e8b12-8806-42cd-83b5-70585af71980 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:43:39.983598 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a83e8b12-8806-42cd-83b5-70585af71980 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:43:40.112253 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1210/2434] 10.222.0.30 () {60 vars in 1153 bytes} [Tue Sep 6 04:43:39 2022] POST /identity/v3/projects => generated 373 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 04:43:40.117494 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6113e934-fc90-41ee-b710-8e4c7bdc17be admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:43:40.123921 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:43:40.128085 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6113e934-fc90-41ee-b710-8e4c7bdc17be admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:43:40.188054 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253a6383-dfa3-4ad5-b524-edcb8955cbc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:43:40.188679 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253a6383-dfa3-4ad5-b524-edcb8955cbc0 admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:43:40.188956 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253a6383-dfa3-4ad5-b524-edcb8955cbc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:43:40.189227 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253a6383-dfa3-4ad5-b524-edcb8955cbc0 admin admin] PATH_INFO: `/v3/users` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:43:40.189892 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253a6383-dfa3-4ad5-b524-edcb8955cbc0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:43:40.191867 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253a6383-dfa3-4ad5-b524-edcb8955cbc0 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:43:40.297771 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1225/2435] 10.222.0.30 () {60 vars in 1144 bytes} [Tue Sep 6 04:43:40 2022] POST /identity/v3/users => generated 287 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 04:43:40.302432 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83e8b12-8806-42cd-83b5-70585af71980 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:43:40.313783 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83e8b12-8806-42cd-83b5-70585af71980 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:43:40.321231 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2360e22-5a8d-45ec-b364-1595eff84c89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:43:40.321917 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2360e22-5a8d-45ec-b364-1595eff84c89 admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:43:40.322202 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2360e22-5a8d-45ec-b364-1595eff84c89 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:43:40.322483 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2360e22-5a8d-45ec-b364-1595eff84c89 admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:43:40.323133 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2360e22-5a8d-45ec-b364-1595eff84c89 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:43:40.326678 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2360e22-5a8d-45ec-b364-1595eff84c89 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:43:40.337824 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1211/2436] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Sep 6 04:43:40 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 06 04:43:40.341871 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253a6383-dfa3-4ad5-b524-edcb8955cbc0 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:43:40.352574 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253a6383-dfa3-4ad5-b524-edcb8955cbc0 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:43:40.357399 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a9d08e-cf08-4e06-bf7f-fd63d63d336f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:43:40.358039 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a9d08e-cf08-4e06-bf7f-fd63d63d336f admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:43:40.358324 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a9d08e-cf08-4e06-bf7f-fd63d63d336f admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:43:40.358620 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a9d08e-cf08-4e06-bf7f-fd63d63d336f admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:43:40.359244 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a9d08e-cf08-4e06-bf7f-fd63d63d336f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:43:40.360850 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a9d08e-cf08-4e06-bf7f-fd63d63d336f admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:43:40.373976 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1226/2437] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Sep 6 04:43:40 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 04:43:40.378999 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2360e22-5a8d-45ec-b364-1595eff84c89 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:43:40.389712 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2360e22-5a8d-45ec-b364-1595eff84c89 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:43:40.394461 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb554f20-1c83-4e92-9bf3-2906026c66a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:43:40.395191 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb554f20-1c83-4e92-9bf3-2906026c66a9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:43:40.395477 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb554f20-1c83-4e92-9bf3-2906026c66a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:43:40.395757 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb554f20-1c83-4e92-9bf3-2906026c66a9 admin admin] PATH_INFO: `/v3/projects/c1db46db3dbd47e4b1ad65aabcef17a2/users/29dbb618cef74598bfcde8f2c440cf82/roles/29607b2de39f40d68bb6ceef64b8a732` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:43:40.412600 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb554f20-1c83-4e92-9bf3-2906026c66a9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c1db46db3dbd47e4b1ad65aabcef17a2, user_id=29dbb618cef74598bfcde8f2c440cf82, role_id=29607b2de39f40d68bb6ceef64b8a732)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:43:40.415268 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb554f20-1c83-4e92-9bf3-2906026c66a9 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:43:40.492267 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1212/2438] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Sep 6 04:43:40 2022] PUT /identity/v3/projects/c1db46db3dbd47e4b1ad65aabcef17a2/users/29dbb618cef74598bfcde8f2c440cf82/roles/29607b2de39f40d68bb6ceef64b8a732 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 04:43:40.503659 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a623e74e-aa90-40e7-97d8-ed3335e78af8 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:43:40.504466 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a623e74e-aa90-40e7-97d8-ed3335e78af8 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:43:40.504897 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a623e74e-aa90-40e7-97d8-ed3335e78af8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:43:40.532288 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-a623e74e-aa90-40e7-97d8-ed3335e78af8 None None] MFA Rules not processed for user `29dbb618cef74598bfcde8f2c440cf82`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:43:40.533547 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-a623e74e-aa90-40e7-97d8-ed3335e78af8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:43:40.596587 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1227/2439] 10.222.0.30 () {58 vars in 958 bytes} [Tue Sep 6 04:43:40 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 04:43:40.607662 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb554f20-1c83-4e92-9bf3-2906026c66a9 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:43:40.609338 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-bb554f20-1c83-4e92-9bf3-2906026c66a9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:43:40.730383 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb554f20-1c83-4e92-9bf3-2906026c66a9 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:43:40.736190 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d565e6cc-3257-441b-8dd0-426f490d63c2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9db138b77afa4b068244b8ef715404fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b707733ce65c42edbc1fd95ad0a1c06c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:43:40.736803 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d565e6cc-3257-441b-8dd0-426f490d63c2 service nova] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:43:40.737127 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d565e6cc-3257-441b-8dd0-426f490d63c2 service nova] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:43:40.737629 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d565e6cc-3257-441b-8dd0-426f490d63c2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:43:40.745629 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d565e6cc-3257-441b-8dd0-426f490d63c2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:43:40.747169 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d565e6cc-3257-441b-8dd0-426f490d63c2 service nova] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:43:40.757023 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1213/2440] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Sep 6 04:43:40 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 04:43:41.267962 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a623e74e-aa90-40e7-97d8-ed3335e78af8 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:43:41.269879 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-a623e74e-aa90-40e7-97d8-ed3335e78af8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:43:41.286507 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a623e74e-aa90-40e7-97d8-ed3335e78af8 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:43:41.291928 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66c64d1-fe89-4342-baf3-803b23399679 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9db138b77afa4b068244b8ef715404fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b707733ce65c42edbc1fd95ad0a1c06c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:43:41.292554 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66c64d1-fe89-4342-baf3-803b23399679 service nova] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:43:41.292832 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66c64d1-fe89-4342-baf3-803b23399679 service nova] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:43:41.293103 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66c64d1-fe89-4342-baf3-803b23399679 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:43:41.294866 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-c66c64d1-fe89-4342-baf3-803b23399679 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:43:41.360360 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66c64d1-fe89-4342-baf3-803b23399679 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:43:41.369074 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66c64d1-fe89-4342-baf3-803b23399679 service nova] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:43:41.375215 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1228/2441] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Sep 6 04:43:41 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 04:43:43.892947 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d565e6cc-3257-441b-8dd0-426f490d63c2 service nova] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:43:43.896844 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=92,keystone:DELETE=5,keystone:INSERT=3 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:43:43.956209 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d565e6cc-3257-441b-8dd0-426f490d63c2 service nova] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:43:43.961270 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b50a6c-655e-4a18-8b2b-4f44ac8f46b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:43:43.961921 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b50a6c-655e-4a18-8b2b-4f44ac8f46b5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:43:43.962198 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b50a6c-655e-4a18-8b2b-4f44ac8f46b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:43:43.962468 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b50a6c-655e-4a18-8b2b-4f44ac8f46b5 admin admin] PATH_INFO: `/v3/users/29dbb618cef74598bfcde8f2c440cf82` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:43:43.963571 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b50a6c-655e-4a18-8b2b-4f44ac8f46b5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=29dbb618cef74598bfcde8f2c440cf82)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:43:43.965500 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b50a6c-655e-4a18-8b2b-4f44ac8f46b5 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:43:44.118435 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-14b50a6c-655e-4a18-8b2b-4f44ac8f46b5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '29dbb618cef74598bfcde8f2c440cf82'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:43:44.184665 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-14b50a6c-655e-4a18-8b2b-4f44ac8f46b5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '29dbb618cef74598bfcde8f2c440cf82'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:43:44.185419 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-14b50a6c-655e-4a18-8b2b-4f44ac8f46b5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '29dbb618cef74598bfcde8f2c440cf82'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:43:44.193276 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-14b50a6c-655e-4a18-8b2b-4f44ac8f46b5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '29dbb618cef74598bfcde8f2c440cf82'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:43:44.206906 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-14b50a6c-655e-4a18-8b2b-4f44ac8f46b5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '29dbb618cef74598bfcde8f2c440cf82'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:43:44.213195 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1214/2442] 10.222.0.30 () {58 vars in 1224 bytes} [Tue Sep 6 04:43:43 2022] DELETE /identity/v3/users/29dbb618cef74598bfcde8f2c440cf82 => generated 0 bytes in 321 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 04:43:44.217764 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e2d82f-cc28-49f5-97de-32ba1dc31cfd None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:43:44.218241 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e2d82f-cc28-49f5-97de-32ba1dc31cfd None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:43:44.218341 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e2d82f-cc28-49f5-97de-32ba1dc31cfd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:43:44.238103 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-51e2d82f-cc28-49f5-97de-32ba1dc31cfd None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:43:44.239345 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-51e2d82f-cc28-49f5-97de-32ba1dc31cfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:43:44.244940 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:43:44.299377 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1229/2443] 10.222.0.30 () {58 vars in 958 bytes} [Tue Sep 6 04:43:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 04:43:44.308979 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b50a6c-655e-4a18-8b2b-4f44ac8f46b5 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:43:44.310449 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-14b50a6c-655e-4a18-8b2b-4f44ac8f46b5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:43:44.398781 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b50a6c-655e-4a18-8b2b-4f44ac8f46b5 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:43:44.402694 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f201c096-5b57-4ff0-bae0-dd2510d11292 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90492440f6a6471e8392361c098428aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b707733ce65c42edbc1fd95ad0a1c06c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:43:44.405625 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f201c096-5b57-4ff0-bae0-dd2510d11292 service neutron] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:43:44.405625 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f201c096-5b57-4ff0-bae0-dd2510d11292 service neutron] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:43:44.405625 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f201c096-5b57-4ff0-bae0-dd2510d11292 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:43:44.413465 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f201c096-5b57-4ff0-bae0-dd2510d11292 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:43:44.415127 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f201c096-5b57-4ff0-bae0-dd2510d11292 service neutron] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:43:44.424679 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1215/2444] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Sep 6 04:43:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 04:43:44.820967 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e2d82f-cc28-49f5-97de-32ba1dc31cfd None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:43:44.822442 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-51e2d82f-cc28-49f5-97de-32ba1dc31cfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:43:44.840351 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e2d82f-cc28-49f5-97de-32ba1dc31cfd None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:43:44.847799 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4197cc-2763-462a-a32b-5e7a371b9eaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:43:44.847799 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4197cc-2763-462a-a32b-5e7a371b9eaf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:43:44.847799 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4197cc-2763-462a-a32b-5e7a371b9eaf admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:43:44.847799 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4197cc-2763-462a-a32b-5e7a371b9eaf admin admin] PATH_INFO: `/v3/projects/c1db46db3dbd47e4b1ad65aabcef17a2` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:43:44.848752 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4197cc-2763-462a-a32b-5e7a371b9eaf admin admin] RBAC: Authorizing `identity:delete_project(project_id=c1db46db3dbd47e4b1ad65aabcef17a2)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:43:44.850875 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4197cc-2763-462a-a32b-5e7a371b9eaf admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:43:44.951376 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-ea4197cc-2763-462a-a32b-5e7a371b9eaf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c1db46db3dbd47e4b1ad65aabcef17a2', 'request_id': 'req-ea4197cc-2763-462a-a32b-5e7a371b9eaf'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:43:44.957206 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-ea4197cc-2763-462a-a32b-5e7a371b9eaf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c1db46db3dbd47e4b1ad65aabcef17a2', 'request_id': 'req-ea4197cc-2763-462a-a32b-5e7a371b9eaf'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:43:45.001656 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-ea4197cc-2763-462a-a32b-5e7a371b9eaf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c1db46db3dbd47e4b1ad65aabcef17a2', 'request_id': 'req-ea4197cc-2763-462a-a32b-5e7a371b9eaf'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:43:45.014419 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-ea4197cc-2763-462a-a32b-5e7a371b9eaf admin admin] The token cache is being invalidate because project c1db46db3dbd47e4b1ad65aabcef17a2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81698) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 04:43:45.014711 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-ea4197cc-2763-462a-a32b-5e7a371b9eaf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:43:45.015721 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-ea4197cc-2763-462a-a32b-5e7a371b9eaf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:43:45.017186 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1230/2445] 10.222.0.30 () {58 vars in 1233 bytes} [Tue Sep 6 04:43:44 2022] DELETE /identity/v3/projects/c1db46db3dbd47e4b1ad65aabcef17a2 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 04:43:45.022271 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78e4347-dbf6-436c-bd8e-84648260b48f None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:43:45.022786 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78e4347-dbf6-436c-bd8e-84648260b48f None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:43:45.022880 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78e4347-dbf6-436c-bd8e-84648260b48f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:43:45.031807 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:43:45.043572 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-a78e4347-dbf6-436c-bd8e-84648260b48f None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:43:45.044822 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-a78e4347-dbf6-436c-bd8e-84648260b48f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:43:45.105764 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1216/2446] 10.222.0.30 () {58 vars in 958 bytes} [Tue Sep 6 04:43:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 04:43:45.113967 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9384d8d-401e-4a11-92da-55da696e2d3c None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:43:45.114251 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9384d8d-401e-4a11-92da-55da696e2d3c None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:43:45.114524 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9384d8d-401e-4a11-92da-55da696e2d3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:43:45.134579 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-a9384d8d-401e-4a11-92da-55da696e2d3c None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:43:45.135771 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-a9384d8d-401e-4a11-92da-55da696e2d3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:43:45.143561 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1231/2447] 10.222.0.30 () {58 vars in 958 bytes} [Tue Sep 6 04:43:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 04:43:45.147813 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78e4347-dbf6-436c-bd8e-84648260b48f None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:43:45.157643 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78e4347-dbf6-436c-bd8e-84648260b48f None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:43:45.163242 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3bff60-ff77-4f66-9ead-cec465b98e86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:43:45.163844 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3bff60-ff77-4f66-9ead-cec465b98e86 admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:43:45.164125 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3bff60-ff77-4f66-9ead-cec465b98e86 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:43:45.164396 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3bff60-ff77-4f66-9ead-cec465b98e86 admin admin] PATH_INFO: `/v3/domains` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:43:45.165040 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3bff60-ff77-4f66-9ead-cec465b98e86 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:43:45.165365 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3bff60-ff77-4f66-9ead-cec465b98e86 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:43:45.166946 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3bff60-ff77-4f66-9ead-cec465b98e86 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:43:45.179475 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1217/2448] 10.222.0.30 () {58 vars in 1153 bytes} [Tue Sep 6 04:43:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:43:45.183707 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9384d8d-401e-4a11-92da-55da696e2d3c None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:43:45.193546 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9384d8d-401e-4a11-92da-55da696e2d3c None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:43:45.198317 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858f6aef-926e-42a0-85f5-7bda686f134a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:43:45.198927 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858f6aef-926e-42a0-85f5-7bda686f134a admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:43:45.199209 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858f6aef-926e-42a0-85f5-7bda686f134a admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:43:45.199483 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858f6aef-926e-42a0-85f5-7bda686f134a admin admin] PATH_INFO: `/v3/projects` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:43:45.200124 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858f6aef-926e-42a0-85f5-7bda686f134a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:43:45.202179 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858f6aef-926e-42a0-85f5-7bda686f134a admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:43:45.323556 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1232/2449] 10.222.0.30 () {60 vars in 1153 bytes} [Tue Sep 6 04:43:45 2022] POST /identity/v3/projects => generated 375 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 04:43:45.327983 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3bff60-ff77-4f66-9ead-cec465b98e86 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:43:45.338000 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3bff60-ff77-4f66-9ead-cec465b98e86 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:43:45.395395 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab11722f-f868-4b4d-b09e-05edf47cec7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:43:45.396008 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab11722f-f868-4b4d-b09e-05edf47cec7c admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:43:45.396288 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab11722f-f868-4b4d-b09e-05edf47cec7c admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:43:45.396557 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab11722f-f868-4b4d-b09e-05edf47cec7c admin admin] PATH_INFO: `/v3/users` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:43:45.397190 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab11722f-f868-4b4d-b09e-05edf47cec7c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:43:45.399168 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab11722f-f868-4b4d-b09e-05edf47cec7c admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:43:45.475932 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1218/2450] 10.222.0.30 () {60 vars in 1144 bytes} [Tue Sep 6 04:43:45 2022] POST /identity/v3/users => generated 288 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 04:43:45.480090 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858f6aef-926e-42a0-85f5-7bda686f134a admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:43:45.491007 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858f6aef-926e-42a0-85f5-7bda686f134a admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:43:45.495590 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e17a154-727d-46f6-9862-454c2442b5a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:43:45.496191 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e17a154-727d-46f6-9862-454c2442b5a4 admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:43:45.496467 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e17a154-727d-46f6-9862-454c2442b5a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:43:45.496732 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e17a154-727d-46f6-9862-454c2442b5a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:43:45.497401 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e17a154-727d-46f6-9862-454c2442b5a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:43:45.498984 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e17a154-727d-46f6-9862-454c2442b5a4 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:43:45.508995 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1233/2451] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Sep 6 04:43:45 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 04:43:45.513016 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab11722f-f868-4b4d-b09e-05edf47cec7c admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:43:45.524446 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab11722f-f868-4b4d-b09e-05edf47cec7c admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:43:45.529552 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57085520-85c6-4b8b-a3d7-33a8ab226cec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:43:45.530151 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57085520-85c6-4b8b-a3d7-33a8ab226cec admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:43:45.530431 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57085520-85c6-4b8b-a3d7-33a8ab226cec admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:43:45.530701 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57085520-85c6-4b8b-a3d7-33a8ab226cec admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:43:45.531357 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57085520-85c6-4b8b-a3d7-33a8ab226cec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:43:45.532932 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57085520-85c6-4b8b-a3d7-33a8ab226cec admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:43:45.544523 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1219/2452] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Sep 6 04:43:45 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 04:43:45.548648 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e17a154-727d-46f6-9862-454c2442b5a4 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:43:45.558809 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e17a154-727d-46f6-9862-454c2442b5a4 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:43:45.563419 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671d5e26-93b1-4b16-beb2-e0a50dafbcfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:43:45.564065 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671d5e26-93b1-4b16-beb2-e0a50dafbcfe admin admin] REQUEST_METHOD: `PUT` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:43:45.564347 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671d5e26-93b1-4b16-beb2-e0a50dafbcfe admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:43:45.564622 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671d5e26-93b1-4b16-beb2-e0a50dafbcfe admin admin] PATH_INFO: `/v3/projects/2461d623f18f4813a1ccd342423f2ad0/users/450e190d83dd4d2d8024dbbeb8c9fa03/roles/29607b2de39f40d68bb6ceef64b8a732` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:43:45.579110 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671d5e26-93b1-4b16-beb2-e0a50dafbcfe admin admin] RBAC: Authorizing `identity:create_grant(project_id=2461d623f18f4813a1ccd342423f2ad0, user_id=450e190d83dd4d2d8024dbbeb8c9fa03, role_id=29607b2de39f40d68bb6ceef64b8a732)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:43:45.582506 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671d5e26-93b1-4b16-beb2-e0a50dafbcfe admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:43:45.649432 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1234/2453] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Sep 6 04:43:45 2022] PUT /identity/v3/projects/2461d623f18f4813a1ccd342423f2ad0/users/450e190d83dd4d2d8024dbbeb8c9fa03/roles/29607b2de39f40d68bb6ceef64b8a732 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 04:43:45.662520 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56f6b0d-8106-4382-9d64-169e1c6fc0e3 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:43:45.662857 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56f6b0d-8106-4382-9d64-169e1c6fc0e3 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:43:45.663134 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56f6b0d-8106-4382-9d64-169e1c6fc0e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:43:45.690284 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-c56f6b0d-8106-4382-9d64-169e1c6fc0e3 None None] MFA Rules not processed for user `450e190d83dd4d2d8024dbbeb8c9fa03`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:43:45.691614 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-c56f6b0d-8106-4382-9d64-169e1c6fc0e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:43:45.754398 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:43:45.758016 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1220/2454] 10.222.0.30 () {58 vars in 958 bytes} [Tue Sep 6 04:43:45 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 04:43:45.762827 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671d5e26-93b1-4b16-beb2-e0a50dafbcfe admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:43:45.797361 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:43:45.825972 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671d5e26-93b1-4b16-beb2-e0a50dafbcfe admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:43:45.830800 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc6fea8-d086-43d4-89ca-5faeed126f4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:43:45.834689 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc6fea8-d086-43d4-89ca-5faeed126f4f admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:43:45.834689 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc6fea8-d086-43d4-89ca-5faeed126f4f admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:43:45.834689 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc6fea8-d086-43d4-89ca-5faeed126f4f admin admin] PATH_INFO: `/v3/projects` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:43:45.834689 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc6fea8-d086-43d4-89ca-5faeed126f4f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:43:45.835393 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc6fea8-d086-43d4-89ca-5faeed126f4f admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:43:45.952058 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1235/2455] 10.222.0.30 () {60 vars in 1153 bytes} [Tue Sep 6 04:43:45 2022] POST /identity/v3/projects => generated 377 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 04:43:45.956808 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56f6b0d-8106-4382-9d64-169e1c6fc0e3 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:43:45.968001 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56f6b0d-8106-4382-9d64-169e1c6fc0e3 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:43:46.025437 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f30121-87b1-4d1f-8f16-4c9ae2490f00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:43:46.026085 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f30121-87b1-4d1f-8f16-4c9ae2490f00 admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:43:46.026229 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f30121-87b1-4d1f-8f16-4c9ae2490f00 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:43:46.026496 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f30121-87b1-4d1f-8f16-4c9ae2490f00 admin admin] PATH_INFO: `/v3/users` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:43:46.027111 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f30121-87b1-4d1f-8f16-4c9ae2490f00 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:43:46.029387 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f30121-87b1-4d1f-8f16-4c9ae2490f00 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:43:46.107608 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1221/2456] 10.222.0.30 () {60 vars in 1144 bytes} [Tue Sep 6 04:43:45 2022] POST /identity/v3/users => generated 295 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 04:43:46.108911 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc6fea8-d086-43d4-89ca-5faeed126f4f admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:43:46.119533 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc6fea8-d086-43d4-89ca-5faeed126f4f admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:43:46.124284 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e504c22-27d7-438f-b9dd-50aca9e13413 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:43:46.124900 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e504c22-27d7-438f-b9dd-50aca9e13413 admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:43:46.125184 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e504c22-27d7-438f-b9dd-50aca9e13413 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:43:46.125500 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e504c22-27d7-438f-b9dd-50aca9e13413 admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:43:46.126155 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e504c22-27d7-438f-b9dd-50aca9e13413 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:43:46.127755 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e504c22-27d7-438f-b9dd-50aca9e13413 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:43:46.139216 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1236/2457] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Sep 6 04:43:46 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 04:43:46.145210 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f30121-87b1-4d1f-8f16-4c9ae2490f00 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:43:46.156360 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f30121-87b1-4d1f-8f16-4c9ae2490f00 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:43:46.161811 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c48f11-0799-4631-a9d0-048e48b23a06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:43:46.162478 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c48f11-0799-4631-a9d0-048e48b23a06 admin admin] REQUEST_METHOD: `PUT` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:43:46.162765 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c48f11-0799-4631-a9d0-048e48b23a06 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:43:46.163042 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c48f11-0799-4631-a9d0-048e48b23a06 admin admin] PATH_INFO: `/v3/projects/9c4ff5ce6b8c4c3a97d6a70342c4cf2a/users/8e290504c14c4ea48107eb15b4c19fc6/roles/ea4c814713b149108e488de855b025e4` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:43:46.179193 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c48f11-0799-4631-a9d0-048e48b23a06 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9c4ff5ce6b8c4c3a97d6a70342c4cf2a, user_id=8e290504c14c4ea48107eb15b4c19fc6, role_id=ea4c814713b149108e488de855b025e4)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:43:46.181921 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c48f11-0799-4631-a9d0-048e48b23a06 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:43:46.271890 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1222/2458] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Sep 6 04:43:46 2022] PUT /identity/v3/projects/9c4ff5ce6b8c4c3a97d6a70342c4cf2a/users/8e290504c14c4ea48107eb15b4c19fc6/roles/ea4c814713b149108e488de855b025e4 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 04:43:46.276612 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e504c22-27d7-438f-b9dd-50aca9e13413 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:43:46.287280 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e504c22-27d7-438f-b9dd-50aca9e13413 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:43:46.348029 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729d2741-4cab-4750-972e-7521fce1a163 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:43:46.348644 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729d2741-4cab-4750-972e-7521fce1a163 admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:43:46.348920 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729d2741-4cab-4750-972e-7521fce1a163 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:43:46.349195 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729d2741-4cab-4750-972e-7521fce1a163 admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:43:46.349946 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729d2741-4cab-4750-972e-7521fce1a163 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:43:46.351544 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729d2741-4cab-4750-972e-7521fce1a163 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:43:46.361440 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1237/2459] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Sep 6 04:43:46 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 04:43:46.365895 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c48f11-0799-4631-a9d0-048e48b23a06 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:43:46.376206 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c48f11-0799-4631-a9d0-048e48b23a06 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:43:46.381052 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ecfe42-ecee-4ee9-a1c4-6d3eb5ed2c46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:43:46.381734 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ecfe42-ecee-4ee9-a1c4-6d3eb5ed2c46 admin admin] REQUEST_METHOD: `PUT` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:43:46.382010 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ecfe42-ecee-4ee9-a1c4-6d3eb5ed2c46 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:43:46.382287 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ecfe42-ecee-4ee9-a1c4-6d3eb5ed2c46 admin admin] PATH_INFO: `/v3/projects/9c4ff5ce6b8c4c3a97d6a70342c4cf2a/users/8e290504c14c4ea48107eb15b4c19fc6/roles/ea4c814713b149108e488de855b025e4` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:43:46.384378 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ecfe42-ecee-4ee9-a1c4-6d3eb5ed2c46 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9c4ff5ce6b8c4c3a97d6a70342c4cf2a, user_id=8e290504c14c4ea48107eb15b4c19fc6, role_id=ea4c814713b149108e488de855b025e4)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:43:46.387114 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ecfe42-ecee-4ee9-a1c4-6d3eb5ed2c46 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:43:46.398110 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1223/2460] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Sep 6 04:43:46 2022] PUT /identity/v3/projects/9c4ff5ce6b8c4c3a97d6a70342c4cf2a/users/8e290504c14c4ea48107eb15b4c19fc6/roles/ea4c814713b149108e488de855b025e4 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 04:43:46.408819 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274c790e-2ef6-4423-8fe3-810eaf87f825 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:43:46.409731 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274c790e-2ef6-4423-8fe3-810eaf87f825 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:43:46.410013 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274c790e-2ef6-4423-8fe3-810eaf87f825 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:43:46.438183 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-274c790e-2ef6-4423-8fe3-810eaf87f825 None None] MFA Rules not processed for user `8e290504c14c4ea48107eb15b4c19fc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:43:46.439378 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-274c790e-2ef6-4423-8fe3-810eaf87f825 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:43:46.507484 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1238/2461] 10.222.0.30 () {58 vars in 958 bytes} [Tue Sep 6 04:43:46 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 04:43:46.518219 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ecfe42-ecee-4ee9-a1c4-6d3eb5ed2c46 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:43:46.520701 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-b5ecfe42-ecee-4ee9-a1c4-6d3eb5ed2c46 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:43:46.605176 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ecfe42-ecee-4ee9-a1c4-6d3eb5ed2c46 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:43:46.609123 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ddeadb-a552-45c9-a8c3-c5fb5d6ff141 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd09b5ab675634a2eb4b03755743f7d0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b707733ce65c42edbc1fd95ad0a1c06c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:43:46.609765 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ddeadb-a552-45c9-a8c3-c5fb5d6ff141 service cinder] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:43:46.610044 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ddeadb-a552-45c9-a8c3-c5fb5d6ff141 service cinder] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:43:46.610312 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ddeadb-a552-45c9-a8c3-c5fb5d6ff141 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:43:46.657971 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:43:46.669974 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ddeadb-a552-45c9-a8c3-c5fb5d6ff141 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:43:46.671422 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ddeadb-a552-45c9-a8c3-c5fb5d6ff141 service cinder] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:43:46.680975 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1224/2462] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Sep 6 04:43:46 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 04:43:47.688890 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274c790e-2ef6-4423-8fe3-810eaf87f825 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:43:47.690893 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-274c790e-2ef6-4423-8fe3-810eaf87f825 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:43:47.722403 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:43:47.750968 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274c790e-2ef6-4423-8fe3-810eaf87f825 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:43:47.755264 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d1daf5-191a-46de-8d0b-558f3646845c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90492440f6a6471e8392361c098428aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b707733ce65c42edbc1fd95ad0a1c06c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:43:47.755989 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d1daf5-191a-46de-8d0b-558f3646845c service neutron] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:43:47.756270 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d1daf5-191a-46de-8d0b-558f3646845c service neutron] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:43:47.756542 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d1daf5-191a-46de-8d0b-558f3646845c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:43:47.760840 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-d2d1daf5-191a-46de-8d0b-558f3646845c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:43:47.825871 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d1daf5-191a-46de-8d0b-558f3646845c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:43:47.827309 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d1daf5-191a-46de-8d0b-558f3646845c service neutron] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:43:47.836980 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1239/2463] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Sep 6 04:43:47 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 04:43:54.968768 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ddeadb-a552-45c9-a8c3-c5fb5d6ff141 service cinder] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:43:54.978201 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ddeadb-a552-45c9-a8c3-c5fb5d6ff141 service cinder] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:43:54.982054 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a64d21-48ec-4e16-b98c-944600c1d633 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd09b5ab675634a2eb4b03755743f7d0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b707733ce65c42edbc1fd95ad0a1c06c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:43:54.982682 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a64d21-48ec-4e16-b98c-944600c1d633 service cinder] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:43:54.982958 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a64d21-48ec-4e16-b98c-944600c1d633 service cinder] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:43:54.983233 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a64d21-48ec-4e16-b98c-944600c1d633 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:43:54.993712 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a64d21-48ec-4e16-b98c-944600c1d633 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:43:54.995206 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a64d21-48ec-4e16-b98c-944600c1d633 service cinder] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:43:55.005689 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1225/2464] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Sep 6 04:43:54 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 04:43:57.832647 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=38 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:44:05.000660 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=12 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:44:27.266875 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d1daf5-191a-46de-8d0b-558f3646845c service neutron] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:44:27.329419 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d1daf5-191a-46de-8d0b-558f3646845c service neutron] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:44:27.334016 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bcdf73d-b34a-4b9c-afb0-7f78dc59d8bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:44:27.334727 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcdf73d-b34a-4b9c-afb0-7f78dc59d8bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:44:27.335008 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcdf73d-b34a-4b9c-afb0-7f78dc59d8bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:44:27.335375 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcdf73d-b34a-4b9c-afb0-7f78dc59d8bd admin admin] PATH_INFO: `/v3/users/450e190d83dd4d2d8024dbbeb8c9fa03` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:44:27.336494 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bcdf73d-b34a-4b9c-afb0-7f78dc59d8bd admin admin] RBAC: Authorizing `identity:delete_user(user_id=450e190d83dd4d2d8024dbbeb8c9fa03)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:44:27.338429 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bcdf73d-b34a-4b9c-afb0-7f78dc59d8bd admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:44:27.516870 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-6bcdf73d-b34a-4b9c-afb0-7f78dc59d8bd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '450e190d83dd4d2d8024dbbeb8c9fa03'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:44:27.594516 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-6bcdf73d-b34a-4b9c-afb0-7f78dc59d8bd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '450e190d83dd4d2d8024dbbeb8c9fa03'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:44:27.595139 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-6bcdf73d-b34a-4b9c-afb0-7f78dc59d8bd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '450e190d83dd4d2d8024dbbeb8c9fa03'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:44:27.608120 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-6bcdf73d-b34a-4b9c-afb0-7f78dc59d8bd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '450e190d83dd4d2d8024dbbeb8c9fa03'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:44:27.615077 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-6bcdf73d-b34a-4b9c-afb0-7f78dc59d8bd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '450e190d83dd4d2d8024dbbeb8c9fa03'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:44:27.621640 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1240/2465] 10.222.0.30 () {58 vars in 1224 bytes} [Tue Sep 6 04:44:27 2022] DELETE /identity/v3/users/450e190d83dd4d2d8024dbbeb8c9fa03 => generated 0 bytes in 355 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 04:44:27.626085 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216ba178-4604-41bf-90a2-0ae7e6057d7b None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:44:27.626594 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216ba178-4604-41bf-90a2-0ae7e6057d7b None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:44:27.626718 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216ba178-4604-41bf-90a2-0ae7e6057d7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:44:27.646247 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-216ba178-4604-41bf-90a2-0ae7e6057d7b None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:44:27.647537 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-216ba178-4604-41bf-90a2-0ae7e6057d7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:44:27.705521 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1226/2466] 10.222.0.30 () {58 vars in 958 bytes} [Tue Sep 6 04:44:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 04:44:27.715420 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bcdf73d-b34a-4b9c-afb0-7f78dc59d8bd admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:44:27.716873 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-6bcdf73d-b34a-4b9c-afb0-7f78dc59d8bd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:44:27.773175 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bcdf73d-b34a-4b9c-afb0-7f78dc59d8bd admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:44:27.776684 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5295525-9dbc-4a06-9041-50e6fd1e0580 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90492440f6a6471e8392361c098428aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b707733ce65c42edbc1fd95ad0a1c06c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:44:27.777283 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5295525-9dbc-4a06-9041-50e6fd1e0580 service neutron] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:44:27.777605 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5295525-9dbc-4a06-9041-50e6fd1e0580 service neutron] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:44:27.777855 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5295525-9dbc-4a06-9041-50e6fd1e0580 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:44:27.786921 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5295525-9dbc-4a06-9041-50e6fd1e0580 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:44:27.788854 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5295525-9dbc-4a06-9041-50e6fd1e0580 service neutron] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:44:27.798093 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1241/2467] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Sep 6 04:44:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 04:44:28.196611 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216ba178-4604-41bf-90a2-0ae7e6057d7b None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:44:28.198060 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-216ba178-4604-41bf-90a2-0ae7e6057d7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:44:28.211491 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216ba178-4604-41bf-90a2-0ae7e6057d7b None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:44:28.216184 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673717dd-3237-4643-b466-40102eea3a4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:44:28.216833 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673717dd-3237-4643-b466-40102eea3a4f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:44:28.217115 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673717dd-3237-4643-b466-40102eea3a4f admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:44:28.217426 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673717dd-3237-4643-b466-40102eea3a4f admin admin] PATH_INFO: `/v3/projects/2461d623f18f4813a1ccd342423f2ad0` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:44:28.218704 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673717dd-3237-4643-b466-40102eea3a4f admin admin] RBAC: Authorizing `identity:delete_project(project_id=2461d623f18f4813a1ccd342423f2ad0)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:44:28.220450 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673717dd-3237-4643-b466-40102eea3a4f admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:44:28.338306 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-673717dd-3237-4643-b466-40102eea3a4f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2461d623f18f4813a1ccd342423f2ad0', 'request_id': 'req-673717dd-3237-4643-b466-40102eea3a4f'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:44:28.423356 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-673717dd-3237-4643-b466-40102eea3a4f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2461d623f18f4813a1ccd342423f2ad0', 'request_id': 'req-673717dd-3237-4643-b466-40102eea3a4f'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:44:28.430716 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-673717dd-3237-4643-b466-40102eea3a4f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2461d623f18f4813a1ccd342423f2ad0', 'request_id': 'req-673717dd-3237-4643-b466-40102eea3a4f'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:44:28.441370 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-673717dd-3237-4643-b466-40102eea3a4f admin admin] The token cache is being invalidate because project 2461d623f18f4813a1ccd342423f2ad0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81697) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 04:44:28.441659 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-673717dd-3237-4643-b466-40102eea3a4f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:44:28.446016 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-673717dd-3237-4643-b466-40102eea3a4f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:44:28.446016 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1227/2468] 10.222.0.30 () {58 vars in 1233 bytes} [Tue Sep 6 04:44:28 2022] DELETE /identity/v3/projects/2461d623f18f4813a1ccd342423f2ad0 => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 04:44:28.448405 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5295525-9dbc-4a06-9041-50e6fd1e0580 service neutron] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:44:28.458126 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5295525-9dbc-4a06-9041-50e6fd1e0580 service neutron] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:44:28.477673 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:44:28.513872 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26755db-6c84-4a35-ab57-ceb83560ed87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:44:28.514499 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26755db-6c84-4a35-ab57-ceb83560ed87 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:44:28.514878 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26755db-6c84-4a35-ab57-ceb83560ed87 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:44:28.515051 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26755db-6c84-4a35-ab57-ceb83560ed87 admin admin] PATH_INFO: `/v3/users/8e290504c14c4ea48107eb15b4c19fc6` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:44:28.516163 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26755db-6c84-4a35-ab57-ceb83560ed87 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8e290504c14c4ea48107eb15b4c19fc6)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:44:28.518117 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26755db-6c84-4a35-ab57-ceb83560ed87 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:44:28.676687 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-d26755db-6c84-4a35-ab57-ceb83560ed87 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8e290504c14c4ea48107eb15b4c19fc6'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:44:28.754338 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-d26755db-6c84-4a35-ab57-ceb83560ed87 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8e290504c14c4ea48107eb15b4c19fc6'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:44:28.755050 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-d26755db-6c84-4a35-ab57-ceb83560ed87 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8e290504c14c4ea48107eb15b4c19fc6'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:44:28.768027 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-d26755db-6c84-4a35-ab57-ceb83560ed87 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8e290504c14c4ea48107eb15b4c19fc6'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:44:28.775754 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-d26755db-6c84-4a35-ab57-ceb83560ed87 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8e290504c14c4ea48107eb15b4c19fc6'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:44:28.781938 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1242/2469] 10.222.0.30 () {58 vars in 1224 bytes} [Tue Sep 6 04:44:28 2022] DELETE /identity/v3/users/8e290504c14c4ea48107eb15b4c19fc6 => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 04:44:29.185074 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673717dd-3237-4643-b466-40102eea3a4f admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:44:29.186572 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-673717dd-3237-4643-b466-40102eea3a4f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:44:29.200367 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673717dd-3237-4643-b466-40102eea3a4f admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:44:29.266057 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a3c0b6-bc7d-4482-9486-73e5d3d17d87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:44:29.266701 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a3c0b6-bc7d-4482-9486-73e5d3d17d87 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:44:29.266986 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a3c0b6-bc7d-4482-9486-73e5d3d17d87 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:44:29.267264 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a3c0b6-bc7d-4482-9486-73e5d3d17d87 admin admin] PATH_INFO: `/v3/projects/9c4ff5ce6b8c4c3a97d6a70342c4cf2a` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:44:29.268403 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a3c0b6-bc7d-4482-9486-73e5d3d17d87 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9c4ff5ce6b8c4c3a97d6a70342c4cf2a)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:44:29.270390 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a3c0b6-bc7d-4482-9486-73e5d3d17d87 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:44:29.356193 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:44:29.372496 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-65a3c0b6-bc7d-4482-9486-73e5d3d17d87 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9c4ff5ce6b8c4c3a97d6a70342c4cf2a', 'request_id': 'req-65a3c0b6-bc7d-4482-9486-73e5d3d17d87'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:44:29.445042 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-65a3c0b6-bc7d-4482-9486-73e5d3d17d87 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9c4ff5ce6b8c4c3a97d6a70342c4cf2a', 'request_id': 'req-65a3c0b6-bc7d-4482-9486-73e5d3d17d87'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:44:29.451957 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-65a3c0b6-bc7d-4482-9486-73e5d3d17d87 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9c4ff5ce6b8c4c3a97d6a70342c4cf2a', 'request_id': 'req-65a3c0b6-bc7d-4482-9486-73e5d3d17d87'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:44:29.462697 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-65a3c0b6-bc7d-4482-9486-73e5d3d17d87 admin admin] The token cache is being invalidate because project 9c4ff5ce6b8c4c3a97d6a70342c4cf2a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81697) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 04:44:29.462985 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-65a3c0b6-bc7d-4482-9486-73e5d3d17d87 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:44:29.463611 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-65a3c0b6-bc7d-4482-9486-73e5d3d17d87 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:44:29.465240 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1228/2470] 10.222.0.30 () {58 vars in 1233 bytes} [Tue Sep 6 04:44:29 2022] DELETE /identity/v3/projects/9c4ff5ce6b8c4c3a97d6a70342c4cf2a => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 04:44:29.470280 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6faef563-2c0d-4be4-a5e9-a39d3e8a13d3 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:44:29.475190 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6faef563-2c0d-4be4-a5e9-a39d3e8a13d3 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:44:29.475190 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6faef563-2c0d-4be4-a5e9-a39d3e8a13d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:44:29.491652 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-6faef563-2c0d-4be4-a5e9-a39d3e8a13d3 None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:44:29.492857 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-6faef563-2c0d-4be4-a5e9-a39d3e8a13d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:44:29.551571 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1243/2471] 10.222.0.30 () {58 vars in 958 bytes} [Tue Sep 6 04:44:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 04:44:29.561222 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d62688-9bc0-4740-9acf-40aed16beab3 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:44:29.561538 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d62688-9bc0-4740-9acf-40aed16beab3 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:44:29.561818 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d62688-9bc0-4740-9acf-40aed16beab3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:44:29.581840 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-c9d62688-9bc0-4740-9acf-40aed16beab3 None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:44:29.583050 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-c9d62688-9bc0-4740-9acf-40aed16beab3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:44:29.590915 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1229/2472] 10.222.0.30 () {58 vars in 958 bytes} [Tue Sep 6 04:44:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 04:44:29.595095 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6faef563-2c0d-4be4-a5e9-a39d3e8a13d3 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:44:29.605531 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6faef563-2c0d-4be4-a5e9-a39d3e8a13d3 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:44:29.610225 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f5bf12-6e32-4de5-8ecb-83890b7e49dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:44:29.610864 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f5bf12-6e32-4de5-8ecb-83890b7e49dd admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:44:29.611167 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f5bf12-6e32-4de5-8ecb-83890b7e49dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:44:29.611438 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f5bf12-6e32-4de5-8ecb-83890b7e49dd admin admin] PATH_INFO: `/v3/domains` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:44:29.612095 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f5bf12-6e32-4de5-8ecb-83890b7e49dd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:44:29.612384 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f5bf12-6e32-4de5-8ecb-83890b7e49dd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:44:29.614042 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f5bf12-6e32-4de5-8ecb-83890b7e49dd admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:44:29.626456 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1244/2473] 10.222.0.30 () {58 vars in 1153 bytes} [Tue Sep 6 04:44:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 04:44:29.630683 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d62688-9bc0-4740-9acf-40aed16beab3 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:44:29.640491 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d62688-9bc0-4740-9acf-40aed16beab3 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:44:29.645202 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff748500-dcc1-42c1-a8a6-c36f82a27a6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:44:29.645976 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff748500-dcc1-42c1-a8a6-c36f82a27a6d admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:44:29.646258 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff748500-dcc1-42c1-a8a6-c36f82a27a6d admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:44:29.646581 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff748500-dcc1-42c1-a8a6-c36f82a27a6d admin admin] PATH_INFO: `/v3/projects` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:44:29.647252 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff748500-dcc1-42c1-a8a6-c36f82a27a6d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:44:29.649199 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff748500-dcc1-42c1-a8a6-c36f82a27a6d admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:44:29.797520 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1230/2474] 10.222.0.30 () {60 vars in 1153 bytes} [Tue Sep 6 04:44:29 2022] POST /identity/v3/projects => generated 383 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 04:44:29.801432 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f5bf12-6e32-4de5-8ecb-83890b7e49dd admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:44:29.811535 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f5bf12-6e32-4de5-8ecb-83890b7e49dd admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:44:29.838679 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:44:29.868671 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0ddc86-13a0-4e5e-9784-28d6ccc07cac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:44:29.869278 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0ddc86-13a0-4e5e-9784-28d6ccc07cac admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:44:29.869591 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0ddc86-13a0-4e5e-9784-28d6ccc07cac admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:44:29.869857 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0ddc86-13a0-4e5e-9784-28d6ccc07cac admin admin] PATH_INFO: `/v3/users` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:44:29.870488 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0ddc86-13a0-4e5e-9784-28d6ccc07cac admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:44:29.872417 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0ddc86-13a0-4e5e-9784-28d6ccc07cac admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:44:29.942975 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1245/2475] 10.222.0.30 () {60 vars in 1144 bytes} [Tue Sep 6 04:44:29 2022] POST /identity/v3/users => generated 292 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 04:44:29.947492 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff748500-dcc1-42c1-a8a6-c36f82a27a6d admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:44:29.958034 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff748500-dcc1-42c1-a8a6-c36f82a27a6d admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:44:29.963314 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbd35ad-8bd6-4b69-88c9-73cadabf0aa0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:44:29.963918 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbd35ad-8bd6-4b69-88c9-73cadabf0aa0 admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:44:29.964202 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbd35ad-8bd6-4b69-88c9-73cadabf0aa0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:44:29.964502 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbd35ad-8bd6-4b69-88c9-73cadabf0aa0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:44:29.965262 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cbd35ad-8bd6-4b69-88c9-73cadabf0aa0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:44:29.966880 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cbd35ad-8bd6-4b69-88c9-73cadabf0aa0 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:44:29.977996 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1231/2476] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Sep 6 04:44:29 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 04:44:29.982027 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0ddc86-13a0-4e5e-9784-28d6ccc07cac admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:44:29.992531 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0ddc86-13a0-4e5e-9784-28d6ccc07cac admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:44:29.997218 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e687cb5-c66d-499f-950c-72451317316c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:44:29.997870 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e687cb5-c66d-499f-950c-72451317316c admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:44:29.999392 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e687cb5-c66d-499f-950c-72451317316c admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:44:29.999688 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e687cb5-c66d-499f-950c-72451317316c admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:44:30.000350 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e687cb5-c66d-499f-950c-72451317316c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:44:30.001998 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e687cb5-c66d-499f-950c-72451317316c admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:44:30.014849 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1246/2477] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Sep 6 04:44:29 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 04:44:30.019153 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbd35ad-8bd6-4b69-88c9-73cadabf0aa0 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:44:30.030108 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbd35ad-8bd6-4b69-88c9-73cadabf0aa0 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:44:30.034658 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a743901a-457f-4db5-8f86-733a4b3b5bba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:44:30.035313 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a743901a-457f-4db5-8f86-733a4b3b5bba admin admin] REQUEST_METHOD: `PUT` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:44:30.035591 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a743901a-457f-4db5-8f86-733a4b3b5bba admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:44:30.035867 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a743901a-457f-4db5-8f86-733a4b3b5bba admin admin] PATH_INFO: `/v3/projects/25e8791b5f2248558ae0c743c6a6a34b/users/e9a3b0648940435486a12e354d34e597/roles/29607b2de39f40d68bb6ceef64b8a732` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:44:30.051849 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a743901a-457f-4db5-8f86-733a4b3b5bba admin admin] RBAC: Authorizing `identity:create_grant(project_id=25e8791b5f2248558ae0c743c6a6a34b, user_id=e9a3b0648940435486a12e354d34e597, role_id=29607b2de39f40d68bb6ceef64b8a732)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:44:30.054782 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a743901a-457f-4db5-8f86-733a4b3b5bba admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:44:30.119847 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1232/2478] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Sep 6 04:44:30 2022] PUT /identity/v3/projects/25e8791b5f2248558ae0c743c6a6a34b/users/e9a3b0648940435486a12e354d34e597/roles/29607b2de39f40d68bb6ceef64b8a732 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 04:44:30.134269 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb30b2c1-3899-4544-bd46-20bd86cbfc92 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:44:30.134558 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb30b2c1-3899-4544-bd46-20bd86cbfc92 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:44:30.134836 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb30b2c1-3899-4544-bd46-20bd86cbfc92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:44:30.163625 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-fb30b2c1-3899-4544-bd46-20bd86cbfc92 None None] MFA Rules not processed for user `e9a3b0648940435486a12e354d34e597`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:44:30.164855 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-fb30b2c1-3899-4544-bd46-20bd86cbfc92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:44:30.228077 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1247/2479] 10.222.0.30 () {58 vars in 958 bytes} [Tue Sep 6 04:44:30 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 04:44:30.232563 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a743901a-457f-4db5-8f86-733a4b3b5bba admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:44:30.293094 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a743901a-457f-4db5-8f86-733a4b3b5bba admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:44:30.301693 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550ecfb7-6a52-4726-aba7-bfab3bba6370 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:44:30.302372 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550ecfb7-6a52-4726-aba7-bfab3bba6370 admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:44:30.302623 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550ecfb7-6a52-4726-aba7-bfab3bba6370 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:44:30.302921 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550ecfb7-6a52-4726-aba7-bfab3bba6370 admin admin] PATH_INFO: `/v3/projects` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:44:30.303743 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550ecfb7-6a52-4726-aba7-bfab3bba6370 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:44:30.305687 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550ecfb7-6a52-4726-aba7-bfab3bba6370 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:44:30.436140 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1233/2480] 10.222.0.30 () {60 vars in 1153 bytes} [Tue Sep 6 04:44:30 2022] POST /identity/v3/projects => generated 381 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 04:44:30.440404 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb30b2c1-3899-4544-bd46-20bd86cbfc92 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:44:30.452461 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb30b2c1-3899-4544-bd46-20bd86cbfc92 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:44:30.506927 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf44707f-490e-47a2-8a99-338ed0c93e78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:44:30.507530 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf44707f-490e-47a2-8a99-338ed0c93e78 admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:44:30.507809 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf44707f-490e-47a2-8a99-338ed0c93e78 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:44:30.508076 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf44707f-490e-47a2-8a99-338ed0c93e78 admin admin] PATH_INFO: `/v3/users` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:44:30.508698 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf44707f-490e-47a2-8a99-338ed0c93e78 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:44:30.510634 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf44707f-490e-47a2-8a99-338ed0c93e78 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:44:30.606526 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1248/2481] 10.222.0.30 () {60 vars in 1144 bytes} [Tue Sep 6 04:44:30 2022] POST /identity/v3/users => generated 297 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 04:44:30.610430 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550ecfb7-6a52-4726-aba7-bfab3bba6370 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:44:30.621428 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550ecfb7-6a52-4726-aba7-bfab3bba6370 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:44:30.626046 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2938f1-b54c-4a3f-9461-eabc825e8935 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:44:30.626670 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2938f1-b54c-4a3f-9461-eabc825e8935 admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:44:30.626951 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2938f1-b54c-4a3f-9461-eabc825e8935 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:44:30.627225 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2938f1-b54c-4a3f-9461-eabc825e8935 admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:44:30.627864 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2938f1-b54c-4a3f-9461-eabc825e8935 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:44:30.629484 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2938f1-b54c-4a3f-9461-eabc825e8935 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:44:30.641085 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1234/2482] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Sep 6 04:44:30 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 04:44:30.645224 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf44707f-490e-47a2-8a99-338ed0c93e78 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:44:30.655776 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf44707f-490e-47a2-8a99-338ed0c93e78 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:44:30.660395 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996fbc15-a59d-443f-aa0d-0a4cc01caff3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:44:30.661171 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996fbc15-a59d-443f-aa0d-0a4cc01caff3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:44:30.661490 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996fbc15-a59d-443f-aa0d-0a4cc01caff3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:44:30.661769 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996fbc15-a59d-443f-aa0d-0a4cc01caff3 admin admin] PATH_INFO: `/v3/projects/0bb173b2478d41c492edf2146fd3ca65/users/5a565cdce1ef4f729c7b96cd1c5a2c3d/roles/ea4c814713b149108e488de855b025e4` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:44:30.676765 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-996fbc15-a59d-443f-aa0d-0a4cc01caff3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0bb173b2478d41c492edf2146fd3ca65, user_id=5a565cdce1ef4f729c7b96cd1c5a2c3d, role_id=ea4c814713b149108e488de855b025e4)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:44:30.680011 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-996fbc15-a59d-443f-aa0d-0a4cc01caff3 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:44:30.726016 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1249/2483] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Sep 6 04:44:30 2022] PUT /identity/v3/projects/0bb173b2478d41c492edf2146fd3ca65/users/5a565cdce1ef4f729c7b96cd1c5a2c3d/roles/ea4c814713b149108e488de855b025e4 => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 04:44:30.730022 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2938f1-b54c-4a3f-9461-eabc825e8935 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:44:30.739899 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2938f1-b54c-4a3f-9461-eabc825e8935 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:44:30.762032 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:44:30.798843 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534d8308-734c-4049-9230-3afff35b65bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:44:30.799455 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534d8308-734c-4049-9230-3afff35b65bb admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:44:30.799735 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534d8308-734c-4049-9230-3afff35b65bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:44:30.800006 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534d8308-734c-4049-9230-3afff35b65bb admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:44:30.800640 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-534d8308-734c-4049-9230-3afff35b65bb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:44:30.802443 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-534d8308-734c-4049-9230-3afff35b65bb admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:44:30.813279 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1235/2484] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Sep 6 04:44:30 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 04:44:30.817447 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996fbc15-a59d-443f-aa0d-0a4cc01caff3 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:44:30.823161 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:44:30.828365 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996fbc15-a59d-443f-aa0d-0a4cc01caff3 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:44:30.832875 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d2c410-e9e3-4cdf-a600-17a7a771648f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:44:30.833596 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d2c410-e9e3-4cdf-a600-17a7a771648f admin admin] REQUEST_METHOD: `PUT` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:44:30.833888 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d2c410-e9e3-4cdf-a600-17a7a771648f admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:44:30.834176 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d2c410-e9e3-4cdf-a600-17a7a771648f admin admin] PATH_INFO: `/v3/projects/0bb173b2478d41c492edf2146fd3ca65/users/5a565cdce1ef4f729c7b96cd1c5a2c3d/roles/ea4c814713b149108e488de855b025e4` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:44:30.836083 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d2c410-e9e3-4cdf-a600-17a7a771648f admin admin] RBAC: Authorizing `identity:create_grant(project_id=0bb173b2478d41c492edf2146fd3ca65, user_id=5a565cdce1ef4f729c7b96cd1c5a2c3d, role_id=ea4c814713b149108e488de855b025e4)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:44:30.838734 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d2c410-e9e3-4cdf-a600-17a7a771648f admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:44:30.848961 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1250/2485] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Sep 6 04:44:30 2022] PUT /identity/v3/projects/0bb173b2478d41c492edf2146fd3ca65/users/5a565cdce1ef4f729c7b96cd1c5a2c3d/roles/ea4c814713b149108e488de855b025e4 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 04:44:30.859604 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2787245e-fbd5-43f9-a82d-dde4a55839f4 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:44:30.859833 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2787245e-fbd5-43f9-a82d-dde4a55839f4 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:44:30.860109 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2787245e-fbd5-43f9-a82d-dde4a55839f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:44:30.888333 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-2787245e-fbd5-43f9-a82d-dde4a55839f4 None None] MFA Rules not processed for user `5a565cdce1ef4f729c7b96cd1c5a2c3d`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:44:30.889595 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-2787245e-fbd5-43f9-a82d-dde4a55839f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:44:30.955343 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1236/2486] 10.222.0.30 () {58 vars in 958 bytes} [Tue Sep 6 04:44:30 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 04:44:30.966968 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d2c410-e9e3-4cdf-a600-17a7a771648f admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:44:30.980094 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-d6d2c410-e9e3-4cdf-a600-17a7a771648f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:44:31.027624 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d2c410-e9e3-4cdf-a600-17a7a771648f admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:44:31.031547 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b909b12d-ce0c-4b1f-9977-77610db113eb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd09b5ab675634a2eb4b03755743f7d0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b707733ce65c42edbc1fd95ad0a1c06c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:44:31.032150 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b909b12d-ce0c-4b1f-9977-77610db113eb service cinder] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:44:31.032430 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b909b12d-ce0c-4b1f-9977-77610db113eb service cinder] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:44:31.032704 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b909b12d-ce0c-4b1f-9977-77610db113eb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:44:31.042699 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b909b12d-ce0c-4b1f-9977-77610db113eb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:44:31.044126 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b909b12d-ce0c-4b1f-9977-77610db113eb service cinder] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:44:31.054526 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1251/2487] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Sep 6 04:44:30 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 04:44:31.208092 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2787245e-fbd5-43f9-a82d-dde4a55839f4 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:44:31.218484 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2787245e-fbd5-43f9-a82d-dde4a55839f4 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:44:31.222052 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d9c960-c856-4ea5-8ea6-d8b934e08e4e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd09b5ab675634a2eb4b03755743f7d0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b707733ce65c42edbc1fd95ad0a1c06c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:44:31.222648 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d9c960-c856-4ea5-8ea6-d8b934e08e4e service cinder] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:44:31.222927 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d9c960-c856-4ea5-8ea6-d8b934e08e4e service cinder] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:44:31.223195 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d9c960-c856-4ea5-8ea6-d8b934e08e4e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:44:31.279740 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d9c960-c856-4ea5-8ea6-d8b934e08e4e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:44:31.281183 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d9c960-c856-4ea5-8ea6-d8b934e08e4e service cinder] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:44:31.290597 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1237/2488] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Sep 6 04:44:31 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 04:44:31.717781 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b909b12d-ce0c-4b1f-9977-77610db113eb service cinder] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:44:31.779958 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b909b12d-ce0c-4b1f-9977-77610db113eb service cinder] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:44:31.784405 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6c12f7-2623-4d5b-8f4c-81fcce98a49b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:44:31.785035 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6c12f7-2623-4d5b-8f4c-81fcce98a49b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:44:31.785332 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6c12f7-2623-4d5b-8f4c-81fcce98a49b admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:44:31.785606 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6c12f7-2623-4d5b-8f4c-81fcce98a49b admin admin] PATH_INFO: `/v3/users/e9a3b0648940435486a12e354d34e597` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:44:31.786714 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6c12f7-2623-4d5b-8f4c-81fcce98a49b admin admin] RBAC: Authorizing `identity:delete_user(user_id=e9a3b0648940435486a12e354d34e597)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:44:31.788652 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6c12f7-2623-4d5b-8f4c-81fcce98a49b admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:44:32.019135 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-5a6c12f7-2623-4d5b-8f4c-81fcce98a49b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e9a3b0648940435486a12e354d34e597'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:44:32.079212 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-5a6c12f7-2623-4d5b-8f4c-81fcce98a49b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e9a3b0648940435486a12e354d34e597'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:44:32.079927 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-5a6c12f7-2623-4d5b-8f4c-81fcce98a49b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e9a3b0648940435486a12e354d34e597'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:44:32.093619 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-5a6c12f7-2623-4d5b-8f4c-81fcce98a49b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e9a3b0648940435486a12e354d34e597'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:44:32.100936 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-5a6c12f7-2623-4d5b-8f4c-81fcce98a49b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e9a3b0648940435486a12e354d34e597'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:44:32.106965 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1252/2489] 10.222.0.30 () {58 vars in 1224 bytes} [Tue Sep 6 04:44:31 2022] DELETE /identity/v3/users/e9a3b0648940435486a12e354d34e597 => generated 0 bytes in 390 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 04:44:32.111588 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86a4d45-a5ff-4415-9afd-53a510b9e510 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:44:32.111878 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86a4d45-a5ff-4415-9afd-53a510b9e510 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:44:32.112153 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86a4d45-a5ff-4415-9afd-53a510b9e510 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:44:32.132541 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-b86a4d45-a5ff-4415-9afd-53a510b9e510 None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:44:32.133779 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-b86a4d45-a5ff-4415-9afd-53a510b9e510 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:44:32.173861 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:44:32.175862 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e55c08-3b9f-465c-bf4d-9153b8bec3f4 None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:44:32.176154 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e55c08-3b9f-465c-bf4d-9153b8bec3f4 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:44:32.176438 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e55c08-3b9f-465c-bf4d-9153b8bec3f4 None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:44:32.178348 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1253/2490] 10.222.0.30 () {54 vars in 961 bytes} [Tue Sep 6 04:44:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 04:44:32.183129 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d568ff-db80-4bf2-8a45-38d587555f42 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:44:32.183424 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d568ff-db80-4bf2-8a45-38d587555f42 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:44:32.183706 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d568ff-db80-4bf2-8a45-38d587555f42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:44:32.194464 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1238/2491] 10.222.0.30 () {58 vars in 958 bytes} [Tue Sep 6 04:44:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 04:44:32.201078 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:44:32.203900 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86a4d45-a5ff-4415-9afd-53a510b9e510 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:44:32.205272 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-b86a4d45-a5ff-4415-9afd-53a510b9e510 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:44:32.247124 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-86d568ff-db80-4bf2-8a45-38d587555f42 None None] MFA Rules not processed for user `1d4f2e4dbd41441d84f03ddb30191e7a`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:44:32.248347 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-86d568ff-db80-4bf2-8a45-38d587555f42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:44:32.264190 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86a4d45-a5ff-4415-9afd-53a510b9e510 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:44:32.267867 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb08700-e92e-4036-aec6-6161431942b8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90492440f6a6471e8392361c098428aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b707733ce65c42edbc1fd95ad0a1c06c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:44:32.268456 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb08700-e92e-4036-aec6-6161431942b8 service neutron] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:44:32.268741 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb08700-e92e-4036-aec6-6161431942b8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:44:32.269017 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb08700-e92e-4036-aec6-6161431942b8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:44:32.280887 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb08700-e92e-4036-aec6-6161431942b8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:44:32.280887 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb08700-e92e-4036-aec6-6161431942b8 service neutron] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:44:32.291194 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1239/2492] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Sep 6 04:44:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 06 04:44:32.326915 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1254/2493] 10.222.0.30 () {58 vars in 1058 bytes} [Tue Sep 6 04:44:32 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 04:44:32.335787 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb08700-e92e-4036-aec6-6161431942b8 service neutron] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:44:32.347036 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb08700-e92e-4036-aec6-6161431942b8 service neutron] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:44:32.352628 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131bcac3-94cb-408c-9e18-13d5bfbb56a3 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d4f2e4dbd41441d84f03ddb30191e7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b707733ce65c42edbc1fd95ad0a1c06c', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:44:32.353271 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131bcac3-94cb-408c-9e18-13d5bfbb56a3 service placement] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:44:32.353596 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131bcac3-94cb-408c-9e18-13d5bfbb56a3 service placement] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:44:32.353993 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131bcac3-94cb-408c-9e18-13d5bfbb56a3 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:44:32.355856 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-131bcac3-94cb-408c-9e18-13d5bfbb56a3 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:44:32.368196 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131bcac3-94cb-408c-9e18-13d5bfbb56a3 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:44:32.369840 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131bcac3-94cb-408c-9e18-13d5bfbb56a3 service placement] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:44:32.388230 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1240/2494] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Sep 6 04:44:32 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 06 04:44:32.676839 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d568ff-db80-4bf2-8a45-38d587555f42 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:44:32.678965 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-86d568ff-db80-4bf2-8a45-38d587555f42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:44:32.693191 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d568ff-db80-4bf2-8a45-38d587555f42 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:44:32.698945 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d37c3e-996b-4616-9334-d10a1fea4e9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:44:32.701987 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d37c3e-996b-4616-9334-d10a1fea4e9c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:44:32.701987 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d37c3e-996b-4616-9334-d10a1fea4e9c admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:44:32.701987 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d37c3e-996b-4616-9334-d10a1fea4e9c admin admin] PATH_INFO: `/v3/projects/25e8791b5f2248558ae0c743c6a6a34b` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:44:32.701987 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d37c3e-996b-4616-9334-d10a1fea4e9c admin admin] RBAC: Authorizing `identity:delete_project(project_id=25e8791b5f2248558ae0c743c6a6a34b)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:44:32.703706 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d37c3e-996b-4616-9334-d10a1fea4e9c admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:44:32.818490 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-b1d37c3e-996b-4616-9334-d10a1fea4e9c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '25e8791b5f2248558ae0c743c6a6a34b', 'request_id': 'req-b1d37c3e-996b-4616-9334-d10a1fea4e9c'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:44:32.825026 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-b1d37c3e-996b-4616-9334-d10a1fea4e9c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '25e8791b5f2248558ae0c743c6a6a34b', 'request_id': 'req-b1d37c3e-996b-4616-9334-d10a1fea4e9c'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:44:32.886286 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-b1d37c3e-996b-4616-9334-d10a1fea4e9c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '25e8791b5f2248558ae0c743c6a6a34b', 'request_id': 'req-b1d37c3e-996b-4616-9334-d10a1fea4e9c'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:44:32.898306 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-b1d37c3e-996b-4616-9334-d10a1fea4e9c admin admin] The token cache is being invalidate because project 25e8791b5f2248558ae0c743c6a6a34b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81698) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 04:44:32.898594 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-b1d37c3e-996b-4616-9334-d10a1fea4e9c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:44:32.899216 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-b1d37c3e-996b-4616-9334-d10a1fea4e9c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:44:32.900681 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1255/2495] 10.222.0.30 () {58 vars in 1233 bytes} [Tue Sep 6 04:44:32 2022] DELETE /identity/v3/projects/25e8791b5f2248558ae0c743c6a6a34b => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 04:44:32.904949 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131bcac3-94cb-408c-9e18-13d5bfbb56a3 service placement] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:44:32.916447 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131bcac3-94cb-408c-9e18-13d5bfbb56a3 service placement] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:44:32.971863 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352834a9-3d97-4a67-8534-4d22c3871958 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:44:32.972490 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352834a9-3d97-4a67-8534-4d22c3871958 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:44:32.972769 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352834a9-3d97-4a67-8534-4d22c3871958 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:44:32.973042 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352834a9-3d97-4a67-8534-4d22c3871958 admin admin] PATH_INFO: `/v3/users/5a565cdce1ef4f729c7b96cd1c5a2c3d` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:44:32.974186 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352834a9-3d97-4a67-8534-4d22c3871958 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5a565cdce1ef4f729c7b96cd1c5a2c3d)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:44:32.977071 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352834a9-3d97-4a67-8534-4d22c3871958 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:44:33.146034 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-352834a9-3d97-4a67-8534-4d22c3871958 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5a565cdce1ef4f729c7b96cd1c5a2c3d'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:44:33.223199 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-352834a9-3d97-4a67-8534-4d22c3871958 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5a565cdce1ef4f729c7b96cd1c5a2c3d'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:44:33.223816 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-352834a9-3d97-4a67-8534-4d22c3871958 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5a565cdce1ef4f729c7b96cd1c5a2c3d'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:44:33.231537 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-352834a9-3d97-4a67-8534-4d22c3871958 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5a565cdce1ef4f729c7b96cd1c5a2c3d'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:44:33.241895 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:44:33.244121 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-352834a9-3d97-4a67-8534-4d22c3871958 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5a565cdce1ef4f729c7b96cd1c5a2c3d'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:44:33.250065 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1241/2496] 10.222.0.30 () {58 vars in 1224 bytes} [Tue Sep 6 04:44:32 2022] DELETE /identity/v3/users/5a565cdce1ef4f729c7b96cd1c5a2c3d => generated 0 bytes in 346 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 04:44:33.647256 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d37c3e-996b-4616-9334-d10a1fea4e9c admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:44:33.648857 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-b1d37c3e-996b-4616-9334-d10a1fea4e9c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:44:33.662748 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d37c3e-996b-4616-9334-d10a1fea4e9c admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:44:33.707980 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:44:33.719853 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b59e66-3a88-4035-a22b-361411a05c20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:44:33.720481 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b59e66-3a88-4035-a22b-361411a05c20 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:44:33.720760 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b59e66-3a88-4035-a22b-361411a05c20 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:44:33.721028 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b59e66-3a88-4035-a22b-361411a05c20 admin admin] PATH_INFO: `/v3/projects/0bb173b2478d41c492edf2146fd3ca65` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:44:33.722158 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b59e66-3a88-4035-a22b-361411a05c20 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0bb173b2478d41c492edf2146fd3ca65)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:44:33.724036 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b59e66-3a88-4035-a22b-361411a05c20 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:44:33.811657 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-d8b59e66-3a88-4035-a22b-361411a05c20 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0bb173b2478d41c492edf2146fd3ca65', 'request_id': 'req-d8b59e66-3a88-4035-a22b-361411a05c20'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:44:33.817452 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-d8b59e66-3a88-4035-a22b-361411a05c20 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0bb173b2478d41c492edf2146fd3ca65', 'request_id': 'req-d8b59e66-3a88-4035-a22b-361411a05c20'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:44:33.862953 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-d8b59e66-3a88-4035-a22b-361411a05c20 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0bb173b2478d41c492edf2146fd3ca65', 'request_id': 'req-d8b59e66-3a88-4035-a22b-361411a05c20'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:44:33.874257 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-d8b59e66-3a88-4035-a22b-361411a05c20 admin admin] The token cache is being invalidate because project 0bb173b2478d41c492edf2146fd3ca65 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81698) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 04:44:33.874525 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-d8b59e66-3a88-4035-a22b-361411a05c20 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:44:33.875709 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-d8b59e66-3a88-4035-a22b-361411a05c20 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:44:33.876741 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1256/2497] 10.222.0.30 () {58 vars in 1233 bytes} [Tue Sep 6 04:44:33 2022] DELETE /identity/v3/projects/0bb173b2478d41c492edf2146fd3ca65 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 04:44:33.882818 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48d6f54-2cde-40fb-a967-4ebcf770bb85 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:44:33.883316 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48d6f54-2cde-40fb-a967-4ebcf770bb85 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:44:33.883434 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48d6f54-2cde-40fb-a967-4ebcf770bb85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:44:33.905506 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-b48d6f54-2cde-40fb-a967-4ebcf770bb85 None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:44:33.906732 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-b48d6f54-2cde-40fb-a967-4ebcf770bb85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:44:33.972151 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1242/2498] 10.222.0.30 () {58 vars in 958 bytes} [Tue Sep 6 04:44:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 04:44:33.983008 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077a61cd-36d6-47e9-b7c8-06e3cbd8e307 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:44:33.983498 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077a61cd-36d6-47e9-b7c8-06e3cbd8e307 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:44:33.983984 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077a61cd-36d6-47e9-b7c8-06e3cbd8e307 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:44:34.012823 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-077a61cd-36d6-47e9-b7c8-06e3cbd8e307 None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:44:34.014093 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-077a61cd-36d6-47e9-b7c8-06e3cbd8e307 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:44:34.022170 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1257/2499] 10.222.0.30 () {58 vars in 958 bytes} [Tue Sep 6 04:44:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 04:44:34.027071 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48d6f54-2cde-40fb-a967-4ebcf770bb85 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:44:34.036994 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48d6f54-2cde-40fb-a967-4ebcf770bb85 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:44:34.042074 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537fab2e-acbd-420f-8be6-be7e99bf682c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:44:34.042681 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537fab2e-acbd-420f-8be6-be7e99bf682c admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:44:34.042964 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537fab2e-acbd-420f-8be6-be7e99bf682c admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:44:34.043241 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537fab2e-acbd-420f-8be6-be7e99bf682c admin admin] PATH_INFO: `/v3/domains` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:44:34.043890 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537fab2e-acbd-420f-8be6-be7e99bf682c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:44:34.044194 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537fab2e-acbd-420f-8be6-be7e99bf682c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:44:34.045837 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537fab2e-acbd-420f-8be6-be7e99bf682c admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:44:34.057738 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1243/2500] 10.222.0.30 () {58 vars in 1153 bytes} [Tue Sep 6 04:44:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:44:34.062261 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077a61cd-36d6-47e9-b7c8-06e3cbd8e307 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:44:34.072092 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077a61cd-36d6-47e9-b7c8-06e3cbd8e307 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:44:34.076971 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deafeb97-32d5-4a1b-a12e-a3955c1d3f30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:44:34.077612 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deafeb97-32d5-4a1b-a12e-a3955c1d3f30 admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:44:34.077889 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deafeb97-32d5-4a1b-a12e-a3955c1d3f30 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:44:34.078156 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deafeb97-32d5-4a1b-a12e-a3955c1d3f30 admin admin] PATH_INFO: `/v3/projects` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:44:34.078795 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deafeb97-32d5-4a1b-a12e-a3955c1d3f30 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:44:34.080689 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deafeb97-32d5-4a1b-a12e-a3955c1d3f30 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:44:34.184765 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1258/2501] 10.222.0.30 () {60 vars in 1153 bytes} [Tue Sep 6 04:44:34 2022] POST /identity/v3/projects => generated 369 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 04:44:34.189062 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537fab2e-acbd-420f-8be6-be7e99bf682c admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:44:34.200542 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537fab2e-acbd-420f-8be6-be7e99bf682c admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:44:34.256410 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa3427d-2199-4692-9a41-79d099b0e36f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:44:34.257652 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa3427d-2199-4692-9a41-79d099b0e36f admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:44:34.257929 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa3427d-2199-4692-9a41-79d099b0e36f admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:44:34.258196 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa3427d-2199-4692-9a41-79d099b0e36f admin admin] PATH_INFO: `/v3/users` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:44:34.258823 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa3427d-2199-4692-9a41-79d099b0e36f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:44:34.260710 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa3427d-2199-4692-9a41-79d099b0e36f admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:44:34.343408 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1244/2502] 10.222.0.30 () {60 vars in 1144 bytes} [Tue Sep 6 04:44:34 2022] POST /identity/v3/users => generated 285 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 04:44:34.347654 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deafeb97-32d5-4a1b-a12e-a3955c1d3f30 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:44:34.358531 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deafeb97-32d5-4a1b-a12e-a3955c1d3f30 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:44:34.365200 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e4945e-1cc3-4b63-a609-1519fa517ac1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:44:34.366473 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e4945e-1cc3-4b63-a609-1519fa517ac1 admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:44:34.366744 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e4945e-1cc3-4b63-a609-1519fa517ac1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:44:34.367010 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e4945e-1cc3-4b63-a609-1519fa517ac1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:44:34.367714 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e4945e-1cc3-4b63-a609-1519fa517ac1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:44:34.369390 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e4945e-1cc3-4b63-a609-1519fa517ac1 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:44:34.379780 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1259/2503] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Sep 6 04:44:34 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 04:44:34.384597 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa3427d-2199-4692-9a41-79d099b0e36f admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:44:34.394606 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa3427d-2199-4692-9a41-79d099b0e36f admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:44:34.399886 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d62936e-ac3c-46e5-af39-ceeb2bbd071c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:44:34.400502 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d62936e-ac3c-46e5-af39-ceeb2bbd071c admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:44:34.400782 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d62936e-ac3c-46e5-af39-ceeb2bbd071c admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:44:34.401048 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d62936e-ac3c-46e5-af39-ceeb2bbd071c admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:44:34.401841 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d62936e-ac3c-46e5-af39-ceeb2bbd071c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:44:34.406897 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d62936e-ac3c-46e5-af39-ceeb2bbd071c admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:44:34.415280 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1245/2504] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Sep 6 04:44:34 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 06 04:44:34.419459 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e4945e-1cc3-4b63-a609-1519fa517ac1 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:44:34.430473 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e4945e-1cc3-4b63-a609-1519fa517ac1 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:44:34.435293 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a942b2-274c-48aa-846b-58d0e7458731 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:44:34.435945 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a942b2-274c-48aa-846b-58d0e7458731 admin admin] REQUEST_METHOD: `PUT` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:44:34.436229 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a942b2-274c-48aa-846b-58d0e7458731 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:44:34.436506 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a942b2-274c-48aa-846b-58d0e7458731 admin admin] PATH_INFO: `/v3/projects/39bda01cf822446a9db855d915ac4090/users/433d1c7d1081477f93ef33f58aceba6f/roles/29607b2de39f40d68bb6ceef64b8a732` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:44:34.451988 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a942b2-274c-48aa-846b-58d0e7458731 admin admin] RBAC: Authorizing `identity:create_grant(project_id=39bda01cf822446a9db855d915ac4090, user_id=433d1c7d1081477f93ef33f58aceba6f, role_id=29607b2de39f40d68bb6ceef64b8a732)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:44:34.454694 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a942b2-274c-48aa-846b-58d0e7458731 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:44:34.525985 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1260/2505] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Sep 6 04:44:34 2022] PUT /identity/v3/projects/39bda01cf822446a9db855d915ac4090/users/433d1c7d1081477f93ef33f58aceba6f/roles/29607b2de39f40d68bb6ceef64b8a732 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 04:44:34.531192 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1383e17-d456-469e-b765-a701b01959ef None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:44:34.531480 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1383e17-d456-469e-b765-a701b01959ef None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:44:34.531753 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1383e17-d456-469e-b765-a701b01959ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:44:34.551089 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-c1383e17-d456-469e-b765-a701b01959ef None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:44:34.552462 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-c1383e17-d456-469e-b765-a701b01959ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:44:34.602447 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:DELETE=1,keystone:SELECT=96,keystone:INSERT=3 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:44:34.610809 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1246/2506] 10.222.0.30 () {58 vars in 958 bytes} [Tue Sep 6 04:44:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 04:44:34.621145 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a942b2-274c-48aa-846b-58d0e7458731 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:44:34.622939 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-65a942b2-274c-48aa-846b-58d0e7458731 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:44:34.687435 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a942b2-274c-48aa-846b-58d0e7458731 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:44:34.691114 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f7a027-1401-4959-bf8c-c0fc93fb5e5b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90492440f6a6471e8392361c098428aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b707733ce65c42edbc1fd95ad0a1c06c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:44:34.691723 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f7a027-1401-4959-bf8c-c0fc93fb5e5b service neutron] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:44:34.692006 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f7a027-1401-4959-bf8c-c0fc93fb5e5b service neutron] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:44:34.692284 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f7a027-1401-4959-bf8c-c0fc93fb5e5b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:44:34.701946 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f7a027-1401-4959-bf8c-c0fc93fb5e5b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:44:34.706729 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f7a027-1401-4959-bf8c-c0fc93fb5e5b service neutron] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:44:34.713114 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1261/2507] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Sep 6 04:44:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 04:44:36.252491 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1234aa-c267-4454-98e0-e5a7baa094e2 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:44:36.255714 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1234aa-c267-4454-98e0-e5a7baa094e2 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:44:36.255714 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1234aa-c267-4454-98e0-e5a7baa094e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:44:36.290093 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-be1234aa-c267-4454-98e0-e5a7baa094e2 None None] MFA Rules not processed for user `433d1c7d1081477f93ef33f58aceba6f`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:44:36.290093 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-be1234aa-c267-4454-98e0-e5a7baa094e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:44:36.370745 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1247/2508] 10.222.0.30 () {58 vars in 958 bytes} [Tue Sep 6 04:44:36 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 04:44:36.371945 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f7a027-1401-4959-bf8c-c0fc93fb5e5b service neutron] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:44:36.381876 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f7a027-1401-4959-bf8c-c0fc93fb5e5b service neutron] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:44:36.387558 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3ee162-d016-4c44-b9ac-495741e4594e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:44:36.388206 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3ee162-d016-4c44-b9ac-495741e4594e admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:44:36.388517 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3ee162-d016-4c44-b9ac-495741e4594e admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:44:36.388789 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3ee162-d016-4c44-b9ac-495741e4594e admin admin] PATH_INFO: `/v3/projects` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:44:36.389495 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3ee162-d016-4c44-b9ac-495741e4594e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:44:36.391411 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3ee162-d016-4c44-b9ac-495741e4594e admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:44:36.409600 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:44:36.508685 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1262/2509] 10.222.0.30 () {60 vars in 1153 bytes} [Tue Sep 6 04:44:36 2022] POST /identity/v3/projects => generated 371 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 04:44:36.510273 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1234aa-c267-4454-98e0-e5a7baa094e2 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:44:36.520922 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1234aa-c267-4454-98e0-e5a7baa094e2 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:44:36.578377 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4226fa-3a1f-4c32-84db-241b5d3fc76b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:44:36.578988 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4226fa-3a1f-4c32-84db-241b5d3fc76b admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:44:36.579265 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4226fa-3a1f-4c32-84db-241b5d3fc76b admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:44:36.579535 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4226fa-3a1f-4c32-84db-241b5d3fc76b admin admin] PATH_INFO: `/v3/users` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:44:36.580155 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4226fa-3a1f-4c32-84db-241b5d3fc76b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:44:36.582265 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4226fa-3a1f-4c32-84db-241b5d3fc76b admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:44:36.652216 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1248/2510] 10.222.0.30 () {60 vars in 1144 bytes} [Tue Sep 6 04:44:36 2022] POST /identity/v3/users => generated 292 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 04:44:36.656497 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3ee162-d016-4c44-b9ac-495741e4594e admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:44:36.667418 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3ee162-d016-4c44-b9ac-495741e4594e admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:44:36.672226 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d84bf4-c560-4d86-bc33-66bca1afc359 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:44:36.672831 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d84bf4-c560-4d86-bc33-66bca1afc359 admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:44:36.673134 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d84bf4-c560-4d86-bc33-66bca1afc359 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:44:36.673445 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d84bf4-c560-4d86-bc33-66bca1afc359 admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:44:36.674086 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d84bf4-c560-4d86-bc33-66bca1afc359 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:44:36.675705 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d84bf4-c560-4d86-bc33-66bca1afc359 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:44:36.687093 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1263/2511] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Sep 6 04:44:36 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 06 04:44:36.691243 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4226fa-3a1f-4c32-84db-241b5d3fc76b admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:44:36.701353 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4226fa-3a1f-4c32-84db-241b5d3fc76b admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:44:36.705885 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec50324-80bc-4b25-ac9c-8cf09e73b9f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:44:36.706623 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec50324-80bc-4b25-ac9c-8cf09e73b9f7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:44:36.706900 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec50324-80bc-4b25-ac9c-8cf09e73b9f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:44:36.707170 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec50324-80bc-4b25-ac9c-8cf09e73b9f7 admin admin] PATH_INFO: `/v3/projects/5cc866a5a4af4538a0c4ff03ee73760e/users/9bbcc7f9c4124c84b81d6a3a04f39a7d/roles/ea4c814713b149108e488de855b025e4` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:44:36.721615 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec50324-80bc-4b25-ac9c-8cf09e73b9f7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5cc866a5a4af4538a0c4ff03ee73760e, user_id=9bbcc7f9c4124c84b81d6a3a04f39a7d, role_id=ea4c814713b149108e488de855b025e4)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:44:36.724228 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec50324-80bc-4b25-ac9c-8cf09e73b9f7 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:44:36.805118 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1249/2512] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Sep 6 04:44:36 2022] PUT /identity/v3/projects/5cc866a5a4af4538a0c4ff03ee73760e/users/9bbcc7f9c4124c84b81d6a3a04f39a7d/roles/ea4c814713b149108e488de855b025e4 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 04:44:36.809258 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d84bf4-c560-4d86-bc33-66bca1afc359 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:44:36.819236 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d84bf4-c560-4d86-bc33-66bca1afc359 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:44:36.875323 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac482a6-8a2f-41ff-b8b4-7b71f4765659 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:44:36.875857 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac482a6-8a2f-41ff-b8b4-7b71f4765659 admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:44:36.876144 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac482a6-8a2f-41ff-b8b4-7b71f4765659 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:44:36.876422 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac482a6-8a2f-41ff-b8b4-7b71f4765659 admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:44:36.877058 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac482a6-8a2f-41ff-b8b4-7b71f4765659 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:44:36.878733 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac482a6-8a2f-41ff-b8b4-7b71f4765659 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:44:36.888449 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1264/2513] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Sep 6 04:44:36 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 04:44:36.892611 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec50324-80bc-4b25-ac9c-8cf09e73b9f7 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:44:36.903328 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec50324-80bc-4b25-ac9c-8cf09e73b9f7 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:44:36.908121 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb62c71f-0edb-466a-a0d2-6a6199aa273c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:44:36.908766 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb62c71f-0edb-466a-a0d2-6a6199aa273c admin admin] REQUEST_METHOD: `PUT` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:44:36.909044 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb62c71f-0edb-466a-a0d2-6a6199aa273c admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:44:36.909325 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb62c71f-0edb-466a-a0d2-6a6199aa273c admin admin] PATH_INFO: `/v3/projects/5cc866a5a4af4538a0c4ff03ee73760e/users/9bbcc7f9c4124c84b81d6a3a04f39a7d/roles/ea4c814713b149108e488de855b025e4` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:44:36.911234 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb62c71f-0edb-466a-a0d2-6a6199aa273c admin admin] RBAC: Authorizing `identity:create_grant(project_id=5cc866a5a4af4538a0c4ff03ee73760e, user_id=9bbcc7f9c4124c84b81d6a3a04f39a7d, role_id=ea4c814713b149108e488de855b025e4)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:44:36.913873 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb62c71f-0edb-466a-a0d2-6a6199aa273c admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:44:36.925607 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1250/2514] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Sep 6 04:44:36 2022] PUT /identity/v3/projects/5cc866a5a4af4538a0c4ff03ee73760e/users/9bbcc7f9c4124c84b81d6a3a04f39a7d/roles/ea4c814713b149108e488de855b025e4 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 04:44:38.036805 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a63fc9-9cdf-43cf-a791-54ebebe1deec None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:44:38.037341 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a63fc9-9cdf-43cf-a791-54ebebe1deec None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:44:38.037432 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a63fc9-9cdf-43cf-a791-54ebebe1deec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:44:38.066207 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-a2a63fc9-9cdf-43cf-a791-54ebebe1deec None None] MFA Rules not processed for user `9bbcc7f9c4124c84b81d6a3a04f39a7d`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:44:38.067435 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-a2a63fc9-9cdf-43cf-a791-54ebebe1deec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:44:38.139433 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1265/2515] 10.222.0.30 () {58 vars in 958 bytes} [Tue Sep 6 04:44:38 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 04:44:38.151342 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb62c71f-0edb-466a-a0d2-6a6199aa273c admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:44:38.153031 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-fb62c71f-0edb-466a-a0d2-6a6199aa273c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:44:38.199090 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:44:38.210463 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb62c71f-0edb-466a-a0d2-6a6199aa273c admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:44:38.214498 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fbc6833-5c10-44af-ac6f-a6074543f413 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd09b5ab675634a2eb4b03755743f7d0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b707733ce65c42edbc1fd95ad0a1c06c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:44:38.215105 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbc6833-5c10-44af-ac6f-a6074543f413 service cinder] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:44:38.215389 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbc6833-5c10-44af-ac6f-a6074543f413 service cinder] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:44:38.215663 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbc6833-5c10-44af-ac6f-a6074543f413 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:44:38.274398 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fbc6833-5c10-44af-ac6f-a6074543f413 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:44:38.275851 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fbc6833-5c10-44af-ac6f-a6074543f413 service cinder] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:44:38.284915 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1251/2516] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Sep 6 04:44:38 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 04:44:41.079899 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a63fc9-9cdf-43cf-a791-54ebebe1deec None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:44:41.090187 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a63fc9-9cdf-43cf-a791-54ebebe1deec None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:44:41.093741 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3da76a-8b98-4ee6-9d19-c58a325567b9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd09b5ab675634a2eb4b03755743f7d0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b707733ce65c42edbc1fd95ad0a1c06c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:44:41.094340 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3da76a-8b98-4ee6-9d19-c58a325567b9 service cinder] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:44:41.094624 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3da76a-8b98-4ee6-9d19-c58a325567b9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:44:41.094897 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3da76a-8b98-4ee6-9d19-c58a325567b9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:44:41.103938 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3da76a-8b98-4ee6-9d19-c58a325567b9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:44:41.105409 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3da76a-8b98-4ee6-9d19-c58a325567b9 service cinder] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:44:41.114779 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1266/2517] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Sep 6 04:44:41 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 04:44:48.280956 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=29 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:44:51.110544 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=76 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:45:01.459569 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fbc6833-5c10-44af-ac6f-a6074543f413 service cinder] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:45:01.537896 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fbc6833-5c10-44af-ac6f-a6074543f413 service cinder] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:45:01.543704 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867a8f60-0fb7-43b9-9ab4-dc47b68857d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:45:01.544355 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867a8f60-0fb7-43b9-9ab4-dc47b68857d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:45:01.544635 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867a8f60-0fb7-43b9-9ab4-dc47b68857d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:45:01.544910 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867a8f60-0fb7-43b9-9ab4-dc47b68857d4 admin admin] PATH_INFO: `/v3/users/433d1c7d1081477f93ef33f58aceba6f` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:45:01.546132 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867a8f60-0fb7-43b9-9ab4-dc47b68857d4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=433d1c7d1081477f93ef33f58aceba6f)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:45:01.548136 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867a8f60-0fb7-43b9-9ab4-dc47b68857d4 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:45:01.689093 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-867a8f60-0fb7-43b9-9ab4-dc47b68857d4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '433d1c7d1081477f93ef33f58aceba6f'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:45:01.765436 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-867a8f60-0fb7-43b9-9ab4-dc47b68857d4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '433d1c7d1081477f93ef33f58aceba6f'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:45:01.766506 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-867a8f60-0fb7-43b9-9ab4-dc47b68857d4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '433d1c7d1081477f93ef33f58aceba6f'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:45:01.773657 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-867a8f60-0fb7-43b9-9ab4-dc47b68857d4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '433d1c7d1081477f93ef33f58aceba6f'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:45:01.786065 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-867a8f60-0fb7-43b9-9ab4-dc47b68857d4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '433d1c7d1081477f93ef33f58aceba6f'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:45:01.792711 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1252/2518] 10.222.0.30 () {58 vars in 1224 bytes} [Tue Sep 6 04:45:01 2022] DELETE /identity/v3/users/433d1c7d1081477f93ef33f58aceba6f => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 04:45:02.069218 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3da76a-8b98-4ee6-9d19-c58a325567b9 service cinder] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:45:02.080209 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3da76a-8b98-4ee6-9d19-c58a325567b9 service cinder] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:45:02.144442 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2549ee12-b753-47a9-9d07-010b590f0ab1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:45:02.145113 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2549ee12-b753-47a9-9d07-010b590f0ab1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:45:02.145425 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2549ee12-b753-47a9-9d07-010b590f0ab1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:45:02.145701 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2549ee12-b753-47a9-9d07-010b590f0ab1 admin admin] PATH_INFO: `/v3/projects/39bda01cf822446a9db855d915ac4090` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:45:02.146828 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2549ee12-b753-47a9-9d07-010b590f0ab1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=39bda01cf822446a9db855d915ac4090)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:45:02.148753 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2549ee12-b753-47a9-9d07-010b590f0ab1 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:45:02.268174 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-2549ee12-b753-47a9-9d07-010b590f0ab1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '39bda01cf822446a9db855d915ac4090', 'request_id': 'req-2549ee12-b753-47a9-9d07-010b590f0ab1'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:45:02.273621 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-2549ee12-b753-47a9-9d07-010b590f0ab1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '39bda01cf822446a9db855d915ac4090', 'request_id': 'req-2549ee12-b753-47a9-9d07-010b590f0ab1'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:45:02.353754 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-2549ee12-b753-47a9-9d07-010b590f0ab1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '39bda01cf822446a9db855d915ac4090', 'request_id': 'req-2549ee12-b753-47a9-9d07-010b590f0ab1'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:45:02.373091 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-2549ee12-b753-47a9-9d07-010b590f0ab1 admin admin] The token cache is being invalidate because project 39bda01cf822446a9db855d915ac4090 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81698) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 04:45:02.373527 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-2549ee12-b753-47a9-9d07-010b590f0ab1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:45:02.374431 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-2549ee12-b753-47a9-9d07-010b590f0ab1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:45:02.376123 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1267/2519] 10.222.0.30 () {58 vars in 1233 bytes} [Tue Sep 6 04:45:02 2022] DELETE /identity/v3/projects/39bda01cf822446a9db855d915ac4090 => generated 0 bytes in 307 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 04:45:02.380502 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867a8f60-0fb7-43b9-9ab4-dc47b68857d4 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:45:02.384739 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-867a8f60-0fb7-43b9-9ab4-dc47b68857d4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:45:02.397242 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867a8f60-0fb7-43b9-9ab4-dc47b68857d4 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:45:02.457211 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f55fdf-7c47-4678-9c9a-11f643fd1941 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:45:02.457981 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f55fdf-7c47-4678-9c9a-11f643fd1941 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:45:02.458336 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f55fdf-7c47-4678-9c9a-11f643fd1941 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:45:02.458617 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f55fdf-7c47-4678-9c9a-11f643fd1941 admin admin] PATH_INFO: `/v3/users/9bbcc7f9c4124c84b81d6a3a04f39a7d` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:45:02.459821 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f55fdf-7c47-4678-9c9a-11f643fd1941 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9bbcc7f9c4124c84b81d6a3a04f39a7d)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:45:02.461816 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f55fdf-7c47-4678-9c9a-11f643fd1941 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:45:02.485444 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:45:02.647735 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-68f55fdf-7c47-4678-9c9a-11f643fd1941 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9bbcc7f9c4124c84b81d6a3a04f39a7d'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:45:02.702495 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-68f55fdf-7c47-4678-9c9a-11f643fd1941 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9bbcc7f9c4124c84b81d6a3a04f39a7d'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:45:02.703153 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-68f55fdf-7c47-4678-9c9a-11f643fd1941 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9bbcc7f9c4124c84b81d6a3a04f39a7d'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:45:02.710521 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-68f55fdf-7c47-4678-9c9a-11f643fd1941 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9bbcc7f9c4124c84b81d6a3a04f39a7d'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:45:02.722930 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-68f55fdf-7c47-4678-9c9a-11f643fd1941 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9bbcc7f9c4124c84b81d6a3a04f39a7d'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:45:02.728929 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1253/2520] 10.222.0.30 () {58 vars in 1224 bytes} [Tue Sep 6 04:45:02 2022] DELETE /identity/v3/users/9bbcc7f9c4124c84b81d6a3a04f39a7d => generated 0 bytes in 349 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 04:45:02.990282 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2549ee12-b753-47a9-9d07-010b590f0ab1 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:45:02.992067 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-2549ee12-b753-47a9-9d07-010b590f0ab1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:45:03.007075 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2549ee12-b753-47a9-9d07-010b590f0ab1 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:45:03.067575 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd6713b-2c91-4eb4-ba76-290d98598196 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:45:03.068211 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd6713b-2c91-4eb4-ba76-290d98598196 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:45:03.068492 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd6713b-2c91-4eb4-ba76-290d98598196 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:45:03.068766 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd6713b-2c91-4eb4-ba76-290d98598196 admin admin] PATH_INFO: `/v3/projects/5cc866a5a4af4538a0c4ff03ee73760e` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:45:03.069904 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd6713b-2c91-4eb4-ba76-290d98598196 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5cc866a5a4af4538a0c4ff03ee73760e)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:45:03.071801 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd6713b-2c91-4eb4-ba76-290d98598196 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:45:03.183619 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:45:03.195542 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-4fd6713b-2c91-4eb4-ba76-290d98598196 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5cc866a5a4af4538a0c4ff03ee73760e', 'request_id': 'req-4fd6713b-2c91-4eb4-ba76-290d98598196'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:45:03.201419 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-4fd6713b-2c91-4eb4-ba76-290d98598196 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5cc866a5a4af4538a0c4ff03ee73760e', 'request_id': 'req-4fd6713b-2c91-4eb4-ba76-290d98598196'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:45:03.314510 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-4fd6713b-2c91-4eb4-ba76-290d98598196 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5cc866a5a4af4538a0c4ff03ee73760e', 'request_id': 'req-4fd6713b-2c91-4eb4-ba76-290d98598196'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:45:03.326802 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-4fd6713b-2c91-4eb4-ba76-290d98598196 admin admin] The token cache is being invalidate because project 5cc866a5a4af4538a0c4ff03ee73760e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81698) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 04:45:03.327025 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-4fd6713b-2c91-4eb4-ba76-290d98598196 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:45:03.327644 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-4fd6713b-2c91-4eb4-ba76-290d98598196 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:45:03.329111 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1268/2521] 10.222.0.30 () {58 vars in 1233 bytes} [Tue Sep 6 04:45:02 2022] DELETE /identity/v3/projects/5cc866a5a4af4538a0c4ff03ee73760e => generated 0 bytes in 339 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 04:45:03.335074 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646b8c42-a389-489a-a8e7-7802f7fa9584 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:45:03.335362 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646b8c42-a389-489a-a8e7-7802f7fa9584 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:45:03.335637 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646b8c42-a389-489a-a8e7-7802f7fa9584 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:45:03.356445 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-646b8c42-a389-489a-a8e7-7802f7fa9584 None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:45:03.357747 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-646b8c42-a389-489a-a8e7-7802f7fa9584 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:45:03.423415 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1254/2522] 10.222.0.30 () {58 vars in 958 bytes} [Tue Sep 6 04:45:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 04:45:03.431395 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d753da-0f03-40f4-b526-06763af6e6c7 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:45:03.431673 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d753da-0f03-40f4-b526-06763af6e6c7 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:45:03.431954 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d753da-0f03-40f4-b526-06763af6e6c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:45:03.453070 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-f4d753da-0f03-40f4-b526-06763af6e6c7 None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:45:03.454366 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-f4d753da-0f03-40f4-b526-06763af6e6c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:45:03.462434 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1269/2523] 10.222.0.30 () {58 vars in 958 bytes} [Tue Sep 6 04:45:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 04:45:03.467511 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646b8c42-a389-489a-a8e7-7802f7fa9584 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:45:03.477751 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646b8c42-a389-489a-a8e7-7802f7fa9584 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:45:03.482466 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d011bde8-c3d5-4560-ba29-dbd1cb310e37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:45:03.483076 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d011bde8-c3d5-4560-ba29-dbd1cb310e37 admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:45:03.483355 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d011bde8-c3d5-4560-ba29-dbd1cb310e37 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:45:03.483628 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d011bde8-c3d5-4560-ba29-dbd1cb310e37 admin admin] PATH_INFO: `/v3/domains` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:45:03.484324 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d011bde8-c3d5-4560-ba29-dbd1cb310e37 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:45:03.484626 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d011bde8-c3d5-4560-ba29-dbd1cb310e37 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:45:03.486308 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d011bde8-c3d5-4560-ba29-dbd1cb310e37 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:45:03.498865 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1255/2524] 10.222.0.30 () {58 vars in 1153 bytes} [Tue Sep 6 04:45:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 04:45:03.503322 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d753da-0f03-40f4-b526-06763af6e6c7 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:45:03.514549 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d753da-0f03-40f4-b526-06763af6e6c7 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:45:03.521127 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e51f1fc-8a43-40aa-8557-032ab813dfea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:45:03.521846 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e51f1fc-8a43-40aa-8557-032ab813dfea admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:45:03.532627 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e51f1fc-8a43-40aa-8557-032ab813dfea admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:45:03.532627 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e51f1fc-8a43-40aa-8557-032ab813dfea admin admin] PATH_INFO: `/v3/projects` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:45:03.532627 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e51f1fc-8a43-40aa-8557-032ab813dfea admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:45:03.532627 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e51f1fc-8a43-40aa-8557-032ab813dfea admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:45:03.613211 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1270/2525] 10.222.0.30 () {60 vars in 1153 bytes} [Tue Sep 6 04:45:03 2022] POST /identity/v3/projects => generated 373 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 04:45:03.616515 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d011bde8-c3d5-4560-ba29-dbd1cb310e37 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:45:03.627311 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d011bde8-c3d5-4560-ba29-dbd1cb310e37 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:45:03.687963 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96221ae-3b37-4f9c-8985-cef884566eef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:45:03.688629 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96221ae-3b37-4f9c-8985-cef884566eef admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:45:03.688927 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96221ae-3b37-4f9c-8985-cef884566eef admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:45:03.689185 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96221ae-3b37-4f9c-8985-cef884566eef admin admin] PATH_INFO: `/v3/users` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:45:03.689873 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96221ae-3b37-4f9c-8985-cef884566eef admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:45:03.691793 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96221ae-3b37-4f9c-8985-cef884566eef admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:45:03.791393 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1256/2526] 10.222.0.30 () {60 vars in 1144 bytes} [Tue Sep 6 04:45:03 2022] POST /identity/v3/users => generated 287 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 04:45:03.793224 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e51f1fc-8a43-40aa-8557-032ab813dfea admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:45:03.803960 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e51f1fc-8a43-40aa-8557-032ab813dfea admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:45:03.809001 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c053454-e195-4f52-8c95-704dab04d16a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:45:03.809662 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c053454-e195-4f52-8c95-704dab04d16a admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:45:03.809950 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c053454-e195-4f52-8c95-704dab04d16a admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:45:03.810226 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c053454-e195-4f52-8c95-704dab04d16a admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:45:03.810878 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c053454-e195-4f52-8c95-704dab04d16a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:45:03.812554 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c053454-e195-4f52-8c95-704dab04d16a admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:45:03.824342 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1271/2527] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Sep 6 04:45:03 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 04:45:03.828151 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96221ae-3b37-4f9c-8985-cef884566eef admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:45:03.847816 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96221ae-3b37-4f9c-8985-cef884566eef admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:45:03.847816 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e3f908-e0de-4137-b743-c3895dc8e0d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:45:03.847816 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e3f908-e0de-4137-b743-c3895dc8e0d7 admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:45:03.847816 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e3f908-e0de-4137-b743-c3895dc8e0d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:45:03.848399 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e3f908-e0de-4137-b743-c3895dc8e0d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:45:03.848399 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e3f908-e0de-4137-b743-c3895dc8e0d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:45:03.848866 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e3f908-e0de-4137-b743-c3895dc8e0d7 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:45:03.860428 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1257/2528] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Sep 6 04:45:03 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 06 04:45:03.864737 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c053454-e195-4f52-8c95-704dab04d16a admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:45:03.875187 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c053454-e195-4f52-8c95-704dab04d16a admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:45:03.880050 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ed9ac6-9905-499a-a8e3-03981872a7a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:45:03.880699 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ed9ac6-9905-499a-a8e3-03981872a7a5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:45:03.880982 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ed9ac6-9905-499a-a8e3-03981872a7a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:45:03.881257 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ed9ac6-9905-499a-a8e3-03981872a7a5 admin admin] PATH_INFO: `/v3/projects/5f758de124e64932af3cc80ed348f51c/users/84150d0100be498b84f4b9a54525a986/roles/29607b2de39f40d68bb6ceef64b8a732` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:45:03.895983 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ed9ac6-9905-499a-a8e3-03981872a7a5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5f758de124e64932af3cc80ed348f51c, user_id=84150d0100be498b84f4b9a54525a986, role_id=29607b2de39f40d68bb6ceef64b8a732)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:45:03.898678 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ed9ac6-9905-499a-a8e3-03981872a7a5 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:45:03.986067 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1272/2529] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Sep 6 04:45:03 2022] PUT /identity/v3/projects/5f758de124e64932af3cc80ed348f51c/users/84150d0100be498b84f4b9a54525a986/roles/29607b2de39f40d68bb6ceef64b8a732 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 04:45:03.991758 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae8e39d-0ac6-4fd0-be88-db8520b598b7 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:45:03.992042 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae8e39d-0ac6-4fd0-be88-db8520b598b7 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:45:03.992446 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae8e39d-0ac6-4fd0-be88-db8520b598b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:45:04.001410 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:DELETE=7,keystone:SELECT=89,keystone:INSERT=4 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:45:04.013190 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-3ae8e39d-0ac6-4fd0-be88-db8520b598b7 None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:45:04.014513 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-3ae8e39d-0ac6-4fd0-be88-db8520b598b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:45:04.095290 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1258/2530] 10.222.0.30 () {58 vars in 958 bytes} [Tue Sep 6 04:45:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 04:45:04.108712 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ed9ac6-9905-499a-a8e3-03981872a7a5 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:45:04.110558 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-97ed9ac6-9905-499a-a8e3-03981872a7a5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:45:04.177107 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ed9ac6-9905-499a-a8e3-03981872a7a5 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:45:04.181966 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696bf8ca-2ccd-4df2-a3a4-9cf4a708ac85 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90492440f6a6471e8392361c098428aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b707733ce65c42edbc1fd95ad0a1c06c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:45:04.182699 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696bf8ca-2ccd-4df2-a3a4-9cf4a708ac85 service neutron] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:45:04.183094 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696bf8ca-2ccd-4df2-a3a4-9cf4a708ac85 service neutron] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:45:04.183390 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696bf8ca-2ccd-4df2-a3a4-9cf4a708ac85 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:45:04.194767 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696bf8ca-2ccd-4df2-a3a4-9cf4a708ac85 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:45:04.196219 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696bf8ca-2ccd-4df2-a3a4-9cf4a708ac85 service neutron] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:45:04.207815 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1273/2531] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Sep 6 04:45:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 04:45:05.403031 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ba3759-f3fc-42fb-9a1f-80c01f948854 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:45:05.403494 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ba3759-f3fc-42fb-9a1f-80c01f948854 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:45:05.404050 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ba3759-f3fc-42fb-9a1f-80c01f948854 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:45:05.424427 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-d5ba3759-f3fc-42fb-9a1f-80c01f948854 None None] MFA Rules not processed for user `5a0294ea40914bf290902307a787642d`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:45:05.425526 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-d5ba3759-f3fc-42fb-9a1f-80c01f948854 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:45:05.485141 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1259/2532] 10.222.0.30 () {58 vars in 958 bytes} [Tue Sep 6 04:45:05 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 04:45:05.723916 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656fc060-2877-4988-b76a-ef7f014a3506 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:45:05.724449 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656fc060-2877-4988-b76a-ef7f014a3506 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:45:05.724449 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656fc060-2877-4988-b76a-ef7f014a3506 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:45:05.755825 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-656fc060-2877-4988-b76a-ef7f014a3506 None None] MFA Rules not processed for user `84150d0100be498b84f4b9a54525a986`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:45:05.757716 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-656fc060-2877-4988-b76a-ef7f014a3506 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:45:05.847170 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=92,keystone:DELETE=5,keystone:INSERT=3 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:45:05.877562 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1274/2533] 10.222.0.30 () {58 vars in 958 bytes} [Tue Sep 6 04:45:05 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 04:45:05.889571 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ba3759-f3fc-42fb-9a1f-80c01f948854 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:45:05.891246 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-d5ba3759-f3fc-42fb-9a1f-80c01f948854 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:45:05.964364 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ba3759-f3fc-42fb-9a1f-80c01f948854 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:45:05.967978 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8884a858-dfad-4a66-93aa-97c2689f1664 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd09b5ab675634a2eb4b03755743f7d0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b707733ce65c42edbc1fd95ad0a1c06c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:45:05.968597 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8884a858-dfad-4a66-93aa-97c2689f1664 service cinder] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:45:05.968880 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8884a858-dfad-4a66-93aa-97c2689f1664 service cinder] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:45:05.969152 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8884a858-dfad-4a66-93aa-97c2689f1664 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:45:05.977877 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8884a858-dfad-4a66-93aa-97c2689f1664 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:45:05.979308 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8884a858-dfad-4a66-93aa-97c2689f1664 service cinder] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:45:05.988679 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1260/2534] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Sep 6 04:45:05 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 04:45:07.742718 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c342f1-e25c-45db-a1fe-9b4357b87962 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:45:07.743303 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c342f1-e25c-45db-a1fe-9b4357b87962 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:45:07.743453 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c342f1-e25c-45db-a1fe-9b4357b87962 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:45:07.763530 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-b1c342f1-e25c-45db-a1fe-9b4357b87962 None None] MFA Rules not processed for user `84150d0100be498b84f4b9a54525a986`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:45:07.767538 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-b1c342f1-e25c-45db-a1fe-9b4357b87962 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:45:07.771703 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1275/2535] 10.222.0.30 () {58 vars in 958 bytes} [Tue Sep 6 04:45:07 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 04:45:15.984520 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=76 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:45:17.757132 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=8 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:45:18.815403 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8884a858-dfad-4a66-93aa-97c2689f1664 service cinder] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:45:18.825582 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8884a858-dfad-4a66-93aa-97c2689f1664 service cinder] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:45:18.829189 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011e468e-2d5d-4f6a-b4a2-a2187e7c2a92 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90492440f6a6471e8392361c098428aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b707733ce65c42edbc1fd95ad0a1c06c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:45:18.829826 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011e468e-2d5d-4f6a-b4a2-a2187e7c2a92 service neutron] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:45:18.830101 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011e468e-2d5d-4f6a-b4a2-a2187e7c2a92 service neutron] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:45:18.830369 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011e468e-2d5d-4f6a-b4a2-a2187e7c2a92 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:45:18.832069 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-011e468e-2d5d-4f6a-b4a2-a2187e7c2a92 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:45:18.901965 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011e468e-2d5d-4f6a-b4a2-a2187e7c2a92 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:45:18.903427 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011e468e-2d5d-4f6a-b4a2-a2187e7c2a92 service neutron] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:45:18.914629 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1261/2536] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Sep 6 04:45:18 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 04:45:23.856663 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c342f1-e25c-45db-a1fe-9b4357b87962 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:45:23.858866 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-b1c342f1-e25c-45db-a1fe-9b4357b87962 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:45:23.873359 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c342f1-e25c-45db-a1fe-9b4357b87962 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:45:23.877000 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92437030-a201-4cae-830e-7ba1da3fb259 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90492440f6a6471e8392361c098428aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b707733ce65c42edbc1fd95ad0a1c06c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:45:23.877688 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92437030-a201-4cae-830e-7ba1da3fb259 service neutron] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:45:23.878170 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92437030-a201-4cae-830e-7ba1da3fb259 service neutron] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:45:23.878614 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92437030-a201-4cae-830e-7ba1da3fb259 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:45:23.880366 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-92437030-a201-4cae-830e-7ba1da3fb259 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:45:23.895078 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92437030-a201-4cae-830e-7ba1da3fb259 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:45:23.896540 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92437030-a201-4cae-830e-7ba1da3fb259 service neutron] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:45:23.910316 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1276/2537] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Sep 6 04:45:23 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 04:45:28.909687 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=34 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:45:31.060735 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011e468e-2d5d-4f6a-b4a2-a2187e7c2a92 service neutron] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:45:31.070502 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011e468e-2d5d-4f6a-b4a2-a2187e7c2a92 service neutron] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:45:31.074181 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63618c54-e1ec-463c-880f-952c3f6a5ee1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd09b5ab675634a2eb4b03755743f7d0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b707733ce65c42edbc1fd95ad0a1c06c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:45:31.074793 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63618c54-e1ec-463c-880f-952c3f6a5ee1 service cinder] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:45:31.075071 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63618c54-e1ec-463c-880f-952c3f6a5ee1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:45:31.075342 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63618c54-e1ec-463c-880f-952c3f6a5ee1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:45:31.077929 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-63618c54-e1ec-463c-880f-952c3f6a5ee1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:45:31.179202 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63618c54-e1ec-463c-880f-952c3f6a5ee1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:45:31.180677 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63618c54-e1ec-463c-880f-952c3f6a5ee1 service cinder] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:45:31.191602 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1262/2538] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Sep 6 04:45:31 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 04:45:33.904819 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=6 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:45:34.672392 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d3fe7d-636b-442f-8950-22d4b64dfe27 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:45:34.677672 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d3fe7d-636b-442f-8950-22d4b64dfe27 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:45:34.677672 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d3fe7d-636b-442f-8950-22d4b64dfe27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:45:34.697848 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-20d3fe7d-636b-442f-8950-22d4b64dfe27 None None] MFA Rules not processed for user `84150d0100be498b84f4b9a54525a986`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:45:34.699279 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-20d3fe7d-636b-442f-8950-22d4b64dfe27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:45:34.707419 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1277/2539] 10.222.0.30 () {58 vars in 958 bytes} [Tue Sep 6 04:45:34 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 04:45:37.802043 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708c1943-10ab-4753-8902-955f2cb14fd7 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:45:37.802572 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708c1943-10ab-4753-8902-955f2cb14fd7 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:45:37.802717 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708c1943-10ab-4753-8902-955f2cb14fd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:45:37.822687 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-708c1943-10ab-4753-8902-955f2cb14fd7 None None] MFA Rules not processed for user `4b6c761af9c5457ea1e481ce400aded8`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:45:37.823946 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-708c1943-10ab-4753-8902-955f2cb14fd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:45:37.884128 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1263/2540] 10.222.0.30 () {58 vars in 958 bytes} [Tue Sep 6 04:45:37 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 04:45:44.688296 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=3 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:45:46.527891 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d3fe7d-636b-442f-8950-22d4b64dfe27 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:45:46.538611 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d3fe7d-636b-442f-8950-22d4b64dfe27 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:45:46.542657 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80fd16ab-cb7b-4b50-8eff-7f7d3ba5163f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90492440f6a6471e8392361c098428aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b707733ce65c42edbc1fd95ad0a1c06c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:45:46.543268 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fd16ab-cb7b-4b50-8eff-7f7d3ba5163f service neutron] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:45:46.543547 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fd16ab-cb7b-4b50-8eff-7f7d3ba5163f service neutron] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:45:46.549261 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fd16ab-cb7b-4b50-8eff-7f7d3ba5163f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:45:46.549261 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-80fd16ab-cb7b-4b50-8eff-7f7d3ba5163f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:45:46.557140 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80fd16ab-cb7b-4b50-8eff-7f7d3ba5163f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:45:46.558966 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80fd16ab-cb7b-4b50-8eff-7f7d3ba5163f service neutron] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:45:46.567540 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1278/2541] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Sep 6 04:45:46 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 04:45:47.877361 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=62 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:45:53.734532 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7525786f-ad51-46e7-b41b-ef6649733efb None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:45:53.735146 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7525786f-ad51-46e7-b41b-ef6649733efb None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:45:53.735146 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7525786f-ad51-46e7-b41b-ef6649733efb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:45:53.792557 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-7525786f-ad51-46e7-b41b-ef6649733efb None None] MFA Rules not processed for user `da5fed17101a4935bcc38b72caf9d263`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:45:53.793768 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-7525786f-ad51-46e7-b41b-ef6649733efb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:45:53.858789 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1264/2542] 10.222.0.30 () {58 vars in 1009 bytes} [Tue Sep 6 04:45:53 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 04:45:53.899219 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80fd16ab-cb7b-4b50-8eff-7f7d3ba5163f service neutron] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:45:53.900648 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-80fd16ab-cb7b-4b50-8eff-7f7d3ba5163f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:45:53.995249 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80fd16ab-cb7b-4b50-8eff-7f7d3ba5163f service neutron] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:45:54.000835 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97438b6-f2db-4b55-8817-d00949c03528 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a676ec7f92a144edbeca8e6029ba9ee6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b707733ce65c42edbc1fd95ad0a1c06c', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:45:54.005208 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97438b6-f2db-4b55-8817-d00949c03528 service swift] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:45:54.005208 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97438b6-f2db-4b55-8817-d00949c03528 service swift] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:45:54.005208 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97438b6-f2db-4b55-8817-d00949c03528 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:45:54.014851 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97438b6-f2db-4b55-8817-d00949c03528 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:45:54.016609 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97438b6-f2db-4b55-8817-d00949c03528 service swift] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:45:54.025653 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1279/2543] 10.222.0.30 () {60 vars in 1384 bytes} [Tue Sep 6 04:45:53 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 06 04:46:03.863446 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=40 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:46:04.021980 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=47 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:46:08.727555 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7525786f-ad51-46e7-b41b-ef6649733efb None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:46:08.738588 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7525786f-ad51-46e7-b41b-ef6649733efb None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:46:08.742738 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc2603c-2783-4861-bdef-db4212a2fc7c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90492440f6a6471e8392361c098428aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b707733ce65c42edbc1fd95ad0a1c06c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:46:08.747212 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc2603c-2783-4861-bdef-db4212a2fc7c service neutron] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:46:08.747527 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc2603c-2783-4861-bdef-db4212a2fc7c service neutron] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:46:08.747815 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc2603c-2783-4861-bdef-db4212a2fc7c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:46:08.750899 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-bbc2603c-2783-4861-bdef-db4212a2fc7c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:46:08.766239 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc2603c-2783-4861-bdef-db4212a2fc7c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:46:08.767682 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc2603c-2783-4861-bdef-db4212a2fc7c service neutron] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:46:08.777769 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1265/2544] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Sep 6 04:46:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 04:46:10.854526 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151460a4-f359-4d34-814f-aa55720f51f9 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:46:10.855965 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151460a4-f359-4d34-814f-aa55720f51f9 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:46:10.855965 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151460a4-f359-4d34-814f-aa55720f51f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:46:10.884075 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-151460a4-f359-4d34-814f-aa55720f51f9 None None] MFA Rules not processed for user `da5fed17101a4935bcc38b72caf9d263`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:46:10.885735 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-151460a4-f359-4d34-814f-aa55720f51f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:46:10.894548 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1280/2545] 10.222.0.30 () {58 vars in 1009 bytes} [Tue Sep 6 04:46:10 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 04:46:10.907512 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc2603c-2783-4861-bdef-db4212a2fc7c service neutron] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:46:10.921066 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc2603c-2783-4861-bdef-db4212a2fc7c service neutron] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:46:10.930603 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7382e5cb-81a8-4905-b757-0101995cac6b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a676ec7f92a144edbeca8e6029ba9ee6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b707733ce65c42edbc1fd95ad0a1c06c', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:46:10.931269 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7382e5cb-81a8-4905-b757-0101995cac6b service swift] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:46:10.931552 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7382e5cb-81a8-4905-b757-0101995cac6b service swift] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:46:10.931827 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7382e5cb-81a8-4905-b757-0101995cac6b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:46:10.944084 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7382e5cb-81a8-4905-b757-0101995cac6b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:46:10.946068 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7382e5cb-81a8-4905-b757-0101995cac6b service swift] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:46:10.973025 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1266/2546] 10.222.0.30 () {60 vars in 1384 bytes} [Tue Sep 6 04:46:10 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 66 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 06 04:46:12.846919 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151460a4-f359-4d34-814f-aa55720f51f9 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:46:12.865440 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151460a4-f359-4d34-814f-aa55720f51f9 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:46:12.874310 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29da9260-eca3-47d8-b273-e12b0fb8d85d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90492440f6a6471e8392361c098428aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b707733ce65c42edbc1fd95ad0a1c06c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:46:12.875927 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29da9260-eca3-47d8-b273-e12b0fb8d85d service neutron] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:46:12.877760 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29da9260-eca3-47d8-b273-e12b0fb8d85d service neutron] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:46:12.878863 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29da9260-eca3-47d8-b273-e12b0fb8d85d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:46:12.884573 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-29da9260-eca3-47d8-b273-e12b0fb8d85d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:46:13.049385 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29da9260-eca3-47d8-b273-e12b0fb8d85d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:46:13.051522 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29da9260-eca3-47d8-b273-e12b0fb8d85d service neutron] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:46:13.064515 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1281/2547] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Sep 6 04:46:12 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 218 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 04:46:13.184153 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7382e5cb-81a8-4905-b757-0101995cac6b service swift] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:46:13.186393 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-7382e5cb-81a8-4905-b757-0101995cac6b service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:46:13.321418 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7382e5cb-81a8-4905-b757-0101995cac6b service swift] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:46:13.326719 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6d9910-fb77-4114-a8f7-6aa7e8673eb3 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d4f2e4dbd41441d84f03ddb30191e7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b707733ce65c42edbc1fd95ad0a1c06c', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:46:13.327395 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6d9910-fb77-4114-a8f7-6aa7e8673eb3 service placement] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:46:13.328052 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6d9910-fb77-4114-a8f7-6aa7e8673eb3 service placement] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:46:13.328349 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6d9910-fb77-4114-a8f7-6aa7e8673eb3 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:46:13.330384 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-be6d9910-fb77-4114-a8f7-6aa7e8673eb3 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:46:13.346173 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6d9910-fb77-4114-a8f7-6aa7e8673eb3 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:46:13.348957 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6d9910-fb77-4114-a8f7-6aa7e8673eb3 service placement] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:46:13.367994 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1267/2548] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Sep 6 04:46:13 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 04:46:19.418449 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6e885e-88f6-483d-a09e-08cbe2e0023a None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:46:19.419478 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6e885e-88f6-483d-a09e-08cbe2e0023a None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:46:19.419980 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6e885e-88f6-483d-a09e-08cbe2e0023a None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:46:19.422418 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1282/2549] 10.222.0.30 () {54 vars in 921 bytes} [Tue Sep 6 04:46:19 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 04:46:19.429227 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8881b9ae-7072-4889-95cd-d0d0406bcca8 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:46:19.430241 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8881b9ae-7072-4889-95cd-d0d0406bcca8 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:46:19.430599 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8881b9ae-7072-4889-95cd-d0d0406bcca8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:46:19.555723 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-8881b9ae-7072-4889-95cd-d0d0406bcca8 None None] MFA Rules not processed for user `9db138b77afa4b068244b8ef715404fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:46:19.557385 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-8881b9ae-7072-4889-95cd-d0d0406bcca8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:46:19.569419 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1268/2550] 10.222.0.30 () {58 vars in 1018 bytes} [Tue Sep 6 04:46:19 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 04:46:19.580940 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6e885e-88f6-483d-a09e-08cbe2e0023a None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:46:19.583268 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-0a6e885e-88f6-483d-a09e-08cbe2e0023a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:46:19.733166 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6e885e-88f6-483d-a09e-08cbe2e0023a None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:46:19.737932 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07798b58-e475-4709-8439-237ee5c95912 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae5c31fb1d0342c0937bbc83a415113d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b707733ce65c42edbc1fd95ad0a1c06c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:46:19.738793 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07798b58-e475-4709-8439-237ee5c95912 service glance] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:46:19.739934 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07798b58-e475-4709-8439-237ee5c95912 service glance] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:46:19.740374 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07798b58-e475-4709-8439-237ee5c95912 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:46:19.754312 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07798b58-e475-4709-8439-237ee5c95912 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:46:19.756057 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07798b58-e475-4709-8439-237ee5c95912 service glance] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:46:19.801214 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1283/2551] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Sep 6 04:46:19 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 221 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 04:46:20.070785 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1bdd95-115c-45a7-8a96-c5baefbb86d5 None None] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:46:20.070785 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1bdd95-115c-45a7-8a96-c5baefbb86d5 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:46:20.070785 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1bdd95-115c-45a7-8a96-c5baefbb86d5 None None] PATH_INFO: `/` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:46:20.071494 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1269/2552] 10.222.0.30 () {54 vars in 921 bytes} [Tue Sep 6 04:46:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 04:46:20.079297 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5776847-29a0-455b-8387-fdede0d64ccf None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:46:20.079891 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5776847-29a0-455b-8387-fdede0d64ccf None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:46:20.080497 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5776847-29a0-455b-8387-fdede0d64ccf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:46:20.177142 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-b5776847-29a0-455b-8387-fdede0d64ccf None None] MFA Rules not processed for user `90492440f6a6471e8392361c098428aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:46:20.179095 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-b5776847-29a0-455b-8387-fdede0d64ccf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:46:20.193188 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1284/2553] 10.222.0.30 () {58 vars in 1018 bytes} [Tue Sep 6 04:46:20 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 04:46:20.207781 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1bdd95-115c-45a7-8a96-c5baefbb86d5 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:46:20.229768 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1bdd95-115c-45a7-8a96-c5baefbb86d5 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:46:20.235338 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2207bee6-4976-4d31-8395-a10f4bab38d7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90492440f6a6471e8392361c098428aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b707733ce65c42edbc1fd95ad0a1c06c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:46:20.236215 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2207bee6-4976-4d31-8395-a10f4bab38d7 service neutron] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:46:20.236519 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2207bee6-4976-4d31-8395-a10f4bab38d7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:46:20.236980 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2207bee6-4976-4d31-8395-a10f4bab38d7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:46:20.251855 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2207bee6-4976-4d31-8395-a10f4bab38d7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:46:20.251855 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2207bee6-4976-4d31-8395-a10f4bab38d7 service neutron] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:46:20.269542 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1270/2554] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Sep 6 04:46:20 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 04:46:20.398610 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7eaf0b-a887-43a5-b099-7ec586734ff6 None None] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:46:20.398920 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7eaf0b-a887-43a5-b099-7ec586734ff6 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:46:20.399166 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7eaf0b-a887-43a5-b099-7ec586734ff6 None None] PATH_INFO: `/` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:46:20.400806 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1285/2555] 10.222.0.30 () {54 vars in 921 bytes} [Tue Sep 6 04:46:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 04:46:20.408109 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0030e389-63c2-4a84-9fe9-5cd8f3f6fb4a None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:46:20.408626 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0030e389-63c2-4a84-9fe9-5cd8f3f6fb4a None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:46:20.409084 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0030e389-63c2-4a84-9fe9-5cd8f3f6fb4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:46:20.450571 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-0030e389-63c2-4a84-9fe9-5cd8f3f6fb4a None None] MFA Rules not processed for user `9db138b77afa4b068244b8ef715404fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:46:20.450571 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-0030e389-63c2-4a84-9fe9-5cd8f3f6fb4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:46:20.460118 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1271/2556] 10.222.0.30 () {58 vars in 1018 bytes} [Tue Sep 6 04:46:20 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 04:46:20.476785 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7eaf0b-a887-43a5-b099-7ec586734ff6 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:46:20.491991 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0030e389-63c2-4a84-9fe9-5cd8f3f6fb4a None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:46:20.493529 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-0030e389-63c2-4a84-9fe9-5cd8f3f6fb4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:46:20.496919 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7eaf0b-a887-43a5-b099-7ec586734ff6 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:46:20.513522 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349ae718-c2b5-4cff-a009-be1060a9c47b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90492440f6a6471e8392361c098428aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b707733ce65c42edbc1fd95ad0a1c06c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:46:20.513522 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349ae718-c2b5-4cff-a009-be1060a9c47b service neutron] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:46:20.513522 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349ae718-c2b5-4cff-a009-be1060a9c47b service neutron] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:46:20.513522 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349ae718-c2b5-4cff-a009-be1060a9c47b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:46:20.517281 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0030e389-63c2-4a84-9fe9-5cd8f3f6fb4a None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:46:20.522420 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b93af5-96b8-432d-9ca4-6144a1b4f769 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:46:20.523341 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b93af5-96b8-432d-9ca4-6144a1b4f769 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:46:20.523808 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b93af5-96b8-432d-9ca4-6144a1b4f769 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:46:20.524079 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b93af5-96b8-432d-9ca4-6144a1b4f769 admin admin] PATH_INFO: `/v3/users/65a8d2df1eed492a874ddaae3235edd0` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:46:20.525218 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b93af5-96b8-432d-9ca4-6144a1b4f769 admin admin] RBAC: Authorizing `identity:delete_user(user_id=65a8d2df1eed492a874ddaae3235edd0)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:46:20.527610 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b93af5-96b8-432d-9ca4-6144a1b4f769 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:46:20.528288 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349ae718-c2b5-4cff-a009-be1060a9c47b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:46:20.530063 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349ae718-c2b5-4cff-a009-be1060a9c47b service neutron] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:46:20.550567 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1286/2557] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Sep 6 04:46:20 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 74 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 06 04:46:20.702312 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-d8b93af5-96b8-432d-9ca4-6144a1b4f769 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '65a8d2df1eed492a874ddaae3235edd0'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:46:20.764245 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-d8b93af5-96b8-432d-9ca4-6144a1b4f769 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '65a8d2df1eed492a874ddaae3235edd0'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:46:20.764918 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-d8b93af5-96b8-432d-9ca4-6144a1b4f769 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '65a8d2df1eed492a874ddaae3235edd0'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:46:20.777457 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-d8b93af5-96b8-432d-9ca4-6144a1b4f769 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '65a8d2df1eed492a874ddaae3235edd0'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:46:20.800732 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:46:20.802585 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-d8b93af5-96b8-432d-9ca4-6144a1b4f769 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '65a8d2df1eed492a874ddaae3235edd0'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:46:20.815687 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1272/2558] 10.222.0.30 () {58 vars in 1224 bytes} [Tue Sep 6 04:46:20 2022] DELETE /identity/v3/users/65a8d2df1eed492a874ddaae3235edd0 => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 04:46:21.136316 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349ae718-c2b5-4cff-a009-be1060a9c47b service neutron] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:46:21.150328 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349ae718-c2b5-4cff-a009-be1060a9c47b service neutron] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:46:21.236511 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:46:21.246484 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e353e0d-49f2-4538-87b8-e806bbbce0fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:46:21.247749 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e353e0d-49f2-4538-87b8-e806bbbce0fe admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:46:21.248886 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e353e0d-49f2-4538-87b8-e806bbbce0fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:46:21.251633 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e353e0d-49f2-4538-87b8-e806bbbce0fe admin admin] PATH_INFO: `/v3/projects/17ca991debba43b98c1ad8f0f91c63a9` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:46:21.255339 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e353e0d-49f2-4538-87b8-e806bbbce0fe admin admin] RBAC: Authorizing `identity:delete_project(project_id=17ca991debba43b98c1ad8f0f91c63a9)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:46:21.258426 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e353e0d-49f2-4538-87b8-e806bbbce0fe admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:46:21.412261 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-2e353e0d-49f2-4538-87b8-e806bbbce0fe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '17ca991debba43b98c1ad8f0f91c63a9', 'request_id': 'req-2e353e0d-49f2-4538-87b8-e806bbbce0fe'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:46:21.418999 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-2e353e0d-49f2-4538-87b8-e806bbbce0fe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '17ca991debba43b98c1ad8f0f91c63a9', 'request_id': 'req-2e353e0d-49f2-4538-87b8-e806bbbce0fe'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:46:21.507533 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-2e353e0d-49f2-4538-87b8-e806bbbce0fe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '17ca991debba43b98c1ad8f0f91c63a9', 'request_id': 'req-2e353e0d-49f2-4538-87b8-e806bbbce0fe'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:46:21.525522 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-2e353e0d-49f2-4538-87b8-e806bbbce0fe admin admin] The token cache is being invalidate because project 17ca991debba43b98c1ad8f0f91c63a9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81698) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 04:46:21.526307 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-2e353e0d-49f2-4538-87b8-e806bbbce0fe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:46:21.527346 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-2e353e0d-49f2-4538-87b8-e806bbbce0fe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:46:21.533976 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1287/2559] 10.222.0.30 () {58 vars in 1233 bytes} [Tue Sep 6 04:46:21 2022] DELETE /identity/v3/projects/17ca991debba43b98c1ad8f0f91c63a9 => generated 0 bytes in 394 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 04:46:21.535180 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b93af5-96b8-432d-9ca4-6144a1b4f769 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:46:21.536760 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-d8b93af5-96b8-432d-9ca4-6144a1b4f769 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:46:21.554468 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b93af5-96b8-432d-9ca4-6144a1b4f769 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:46:21.634181 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d07429-0cca-437b-bafb-001155d9f475 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:46:21.634835 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d07429-0cca-437b-bafb-001155d9f475 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:46:21.635114 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d07429-0cca-437b-bafb-001155d9f475 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:46:21.635388 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d07429-0cca-437b-bafb-001155d9f475 admin admin] PATH_INFO: `/v3/users/ce796b9a37ef4bfe9717e00200514a5d` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:46:21.647883 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d07429-0cca-437b-bafb-001155d9f475 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ce796b9a37ef4bfe9717e00200514a5d)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:46:21.647883 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d07429-0cca-437b-bafb-001155d9f475 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:46:21.871235 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-e0d07429-0cca-437b-bafb-001155d9f475 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ce796b9a37ef4bfe9717e00200514a5d'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:46:22.107843 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-e0d07429-0cca-437b-bafb-001155d9f475 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ce796b9a37ef4bfe9717e00200514a5d'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:46:22.109887 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-e0d07429-0cca-437b-bafb-001155d9f475 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ce796b9a37ef4bfe9717e00200514a5d'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:46:22.139362 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-e0d07429-0cca-437b-bafb-001155d9f475 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ce796b9a37ef4bfe9717e00200514a5d'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:46:22.165229 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-e0d07429-0cca-437b-bafb-001155d9f475 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ce796b9a37ef4bfe9717e00200514a5d'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:46:22.202371 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1273/2560] 10.222.0.30 () {58 vars in 1224 bytes} [Tue Sep 6 04:46:21 2022] DELETE /identity/v3/users/ce796b9a37ef4bfe9717e00200514a5d => generated 0 bytes in 662 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 04:46:22.475872 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e353e0d-49f2-4538-87b8-e806bbbce0fe admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:46:22.476956 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-2e353e0d-49f2-4538-87b8-e806bbbce0fe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:46:22.492206 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e353e0d-49f2-4538-87b8-e806bbbce0fe admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:46:22.573112 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125bab59-b7b9-4258-a26b-b92457e25252 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:46:22.574569 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125bab59-b7b9-4258-a26b-b92457e25252 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:46:22.575103 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125bab59-b7b9-4258-a26b-b92457e25252 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:46:22.575449 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125bab59-b7b9-4258-a26b-b92457e25252 admin admin] PATH_INFO: `/v3/projects/88ba913e7d444dd3ac73b85d93127172` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:46:22.577629 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125bab59-b7b9-4258-a26b-b92457e25252 admin admin] RBAC: Authorizing `identity:delete_project(project_id=88ba913e7d444dd3ac73b85d93127172)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:46:22.580043 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125bab59-b7b9-4258-a26b-b92457e25252 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:46:22.727439 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-125bab59-b7b9-4258-a26b-b92457e25252 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '88ba913e7d444dd3ac73b85d93127172', 'request_id': 'req-125bab59-b7b9-4258-a26b-b92457e25252'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:46:22.736345 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-125bab59-b7b9-4258-a26b-b92457e25252 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '88ba913e7d444dd3ac73b85d93127172', 'request_id': 'req-125bab59-b7b9-4258-a26b-b92457e25252'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:46:22.830320 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-125bab59-b7b9-4258-a26b-b92457e25252 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '88ba913e7d444dd3ac73b85d93127172', 'request_id': 'req-125bab59-b7b9-4258-a26b-b92457e25252'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:46:22.848970 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-125bab59-b7b9-4258-a26b-b92457e25252 admin admin] The token cache is being invalidate because project 88ba913e7d444dd3ac73b85d93127172 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81698) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 04:46:22.867890 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-125bab59-b7b9-4258-a26b-b92457e25252 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:46:22.867890 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-125bab59-b7b9-4258-a26b-b92457e25252 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:46:22.869705 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1288/2561] 10.222.0.30 () {58 vars in 1233 bytes} [Tue Sep 6 04:46:22 2022] DELETE /identity/v3/projects/88ba913e7d444dd3ac73b85d93127172 => generated 0 bytes in 378 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 04:46:23.933051 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdfb35a-1726-4552-a6e0-b00c347b7d1f None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:46:23.933720 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdfb35a-1726-4552-a6e0-b00c347b7d1f None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:46:23.933818 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdfb35a-1726-4552-a6e0-b00c347b7d1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:46:23.954034 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-7cdfb35a-1726-4552-a6e0-b00c347b7d1f None None] MFA Rules not processed for user `da5fed17101a4935bcc38b72caf9d263`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:46:23.959915 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-7cdfb35a-1726-4552-a6e0-b00c347b7d1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:46:24.023604 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1274/2562] 10.222.0.30 () {58 vars in 1009 bytes} [Tue Sep 6 04:46:23 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 04:46:24.041085 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125bab59-b7b9-4258-a26b-b92457e25252 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:46:24.042758 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-125bab59-b7b9-4258-a26b-b92457e25252 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:46:24.069036 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:46:24.131626 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125bab59-b7b9-4258-a26b-b92457e25252 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:46:24.137281 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ba7c99-9834-41ce-96b3-32a9b5185272 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a676ec7f92a144edbeca8e6029ba9ee6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b707733ce65c42edbc1fd95ad0a1c06c', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:46:24.138055 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ba7c99-9834-41ce-96b3-32a9b5185272 service swift] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:46:24.138306 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ba7c99-9834-41ce-96b3-32a9b5185272 service swift] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:46:24.138576 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ba7c99-9834-41ce-96b3-32a9b5185272 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:46:24.148942 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ba7c99-9834-41ce-96b3-32a9b5185272 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:46:24.150505 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ba7c99-9834-41ce-96b3-32a9b5185272 service swift] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:46:24.163301 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1289/2563] 10.222.0.30 () {60 vars in 1384 bytes} [Tue Sep 6 04:46:24 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 06 04:46:30.654009 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f313952-679c-4641-b6b7-40a73200ffed None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:46:30.654804 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f313952-679c-4641-b6b7-40a73200ffed None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:46:30.655027 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f313952-679c-4641-b6b7-40a73200ffed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:46:30.676283 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-6f313952-679c-4641-b6b7-40a73200ffed None None] MFA Rules not processed for user `da5fed17101a4935bcc38b72caf9d263`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:46:30.677554 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-6f313952-679c-4641-b6b7-40a73200ffed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:46:30.685139 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1275/2564] 10.222.0.30 () {58 vars in 967 bytes} [Tue Sep 6 04:46:30 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 04:46:30.699634 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ba7c99-9834-41ce-96b3-32a9b5185272 service swift] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:46:30.710608 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ba7c99-9834-41ce-96b3-32a9b5185272 service swift] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:46:30.716236 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3e6af9-58c7-4a1a-9494-4200663f0994 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a676ec7f92a144edbeca8e6029ba9ee6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b707733ce65c42edbc1fd95ad0a1c06c', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:46:30.717083 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3e6af9-58c7-4a1a-9494-4200663f0994 service swift] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:46:30.717414 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3e6af9-58c7-4a1a-9494-4200663f0994 service swift] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:46:30.717692 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3e6af9-58c7-4a1a-9494-4200663f0994 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:46:30.727693 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc3e6af9-58c7-4a1a-9494-4200663f0994 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:46:30.729289 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc3e6af9-58c7-4a1a-9494-4200663f0994 service swift] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:46:30.738287 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1290/2565] 10.222.0.30 () {60 vars in 1384 bytes} [Tue Sep 6 04:46:30 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 06 04:46:35.675048 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f313952-679c-4641-b6b7-40a73200ffed None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:46:35.676526 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-6f313952-679c-4641-b6b7-40a73200ffed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:46:35.697881 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:46:35.755251 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f313952-679c-4641-b6b7-40a73200ffed None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:46:35.760531 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1875b28d-91c6-4f2d-84e9-16231d882818 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:46:35.761167 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1875b28d-91c6-4f2d-84e9-16231d882818 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:46:35.761476 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1875b28d-91c6-4f2d-84e9-16231d882818 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:46:35.761751 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1875b28d-91c6-4f2d-84e9-16231d882818 admin admin] PATH_INFO: `/v3/users/84150d0100be498b84f4b9a54525a986` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:46:35.762893 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1875b28d-91c6-4f2d-84e9-16231d882818 admin admin] RBAC: Authorizing `identity:delete_user(user_id=84150d0100be498b84f4b9a54525a986)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:46:35.764859 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1875b28d-91c6-4f2d-84e9-16231d882818 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:46:35.945329 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-1875b28d-91c6-4f2d-84e9-16231d882818 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '84150d0100be498b84f4b9a54525a986'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:46:36.037542 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-1875b28d-91c6-4f2d-84e9-16231d882818 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '84150d0100be498b84f4b9a54525a986'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:46:36.038350 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-1875b28d-91c6-4f2d-84e9-16231d882818 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '84150d0100be498b84f4b9a54525a986'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:46:36.046553 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-1875b28d-91c6-4f2d-84e9-16231d882818 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '84150d0100be498b84f4b9a54525a986'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:46:36.059305 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-1875b28d-91c6-4f2d-84e9-16231d882818 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '84150d0100be498b84f4b9a54525a986'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:46:36.066935 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1276/2566] 10.222.0.30 () {58 vars in 1224 bytes} [Tue Sep 6 04:46:35 2022] DELETE /identity/v3/users/84150d0100be498b84f4b9a54525a986 => generated 0 bytes in 392 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 04:46:36.384290 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3e6af9-58c7-4a1a-9494-4200663f0994 service swift] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:46:36.395814 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3e6af9-58c7-4a1a-9494-4200663f0994 service swift] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:46:36.460683 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6c93b3-a9b9-4ab9-a811-dddbb7a628a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:46:36.461264 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6c93b3-a9b9-4ab9-a811-dddbb7a628a5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:46:36.461575 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6c93b3-a9b9-4ab9-a811-dddbb7a628a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:46:36.461849 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6c93b3-a9b9-4ab9-a811-dddbb7a628a5 admin admin] PATH_INFO: `/v3/projects/5f758de124e64932af3cc80ed348f51c` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:46:36.462971 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6c93b3-a9b9-4ab9-a811-dddbb7a628a5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5f758de124e64932af3cc80ed348f51c)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:46:36.464934 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6c93b3-a9b9-4ab9-a811-dddbb7a628a5 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:46:36.576396 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-0f6c93b3-a9b9-4ab9-a811-dddbb7a628a5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5f758de124e64932af3cc80ed348f51c', 'request_id': 'req-0f6c93b3-a9b9-4ab9-a811-dddbb7a628a5'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:46:36.582150 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-0f6c93b3-a9b9-4ab9-a811-dddbb7a628a5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5f758de124e64932af3cc80ed348f51c', 'request_id': 'req-0f6c93b3-a9b9-4ab9-a811-dddbb7a628a5'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:46:36.637708 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-0f6c93b3-a9b9-4ab9-a811-dddbb7a628a5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5f758de124e64932af3cc80ed348f51c', 'request_id': 'req-0f6c93b3-a9b9-4ab9-a811-dddbb7a628a5'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:46:36.652153 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-0f6c93b3-a9b9-4ab9-a811-dddbb7a628a5 admin admin] The token cache is being invalidate because project 5f758de124e64932af3cc80ed348f51c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81698) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 04:46:36.652547 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-0f6c93b3-a9b9-4ab9-a811-dddbb7a628a5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:46:36.653262 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-0f6c93b3-a9b9-4ab9-a811-dddbb7a628a5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:46:36.655197 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1291/2567] 10.222.0.30 () {58 vars in 1233 bytes} [Tue Sep 6 04:46:36 2022] DELETE /identity/v3/projects/5f758de124e64932af3cc80ed348f51c => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 04:46:36.660997 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5e4f68-ead0-49a2-b6f4-90c70d706a29 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:46:36.661281 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5e4f68-ead0-49a2-b6f4-90c70d706a29 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:46:36.666655 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5e4f68-ead0-49a2-b6f4-90c70d706a29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:46:36.680883 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-dc5e4f68-ead0-49a2-b6f4-90c70d706a29 None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:46:36.685026 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-dc5e4f68-ead0-49a2-b6f4-90c70d706a29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:46:36.745509 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1277/2568] 10.222.0.30 () {58 vars in 958 bytes} [Tue Sep 6 04:46:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 04:46:36.753228 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3824b24-20c5-42db-8d8f-851c319ee837 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:46:36.753542 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3824b24-20c5-42db-8d8f-851c319ee837 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:46:36.753811 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3824b24-20c5-42db-8d8f-851c319ee837 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:46:36.773612 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-b3824b24-20c5-42db-8d8f-851c319ee837 None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:46:36.774791 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-b3824b24-20c5-42db-8d8f-851c319ee837 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:46:36.783592 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1292/2569] 10.222.0.30 () {58 vars in 958 bytes} [Tue Sep 6 04:46:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 04:46:36.787897 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5e4f68-ead0-49a2-b6f4-90c70d706a29 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:46:36.798098 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5e4f68-ead0-49a2-b6f4-90c70d706a29 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:46:36.802978 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e0dc9c-f6d8-4f9d-87a2-6ea6df528376 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:46:36.803599 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e0dc9c-f6d8-4f9d-87a2-6ea6df528376 admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:46:36.803882 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e0dc9c-f6d8-4f9d-87a2-6ea6df528376 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:46:36.804159 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e0dc9c-f6d8-4f9d-87a2-6ea6df528376 admin admin] PATH_INFO: `/v3/domains` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:46:36.804814 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e0dc9c-f6d8-4f9d-87a2-6ea6df528376 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:46:36.805122 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e0dc9c-f6d8-4f9d-87a2-6ea6df528376 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:46:36.807015 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e0dc9c-f6d8-4f9d-87a2-6ea6df528376 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:46:36.820219 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1278/2570] 10.222.0.30 () {58 vars in 1153 bytes} [Tue Sep 6 04:46:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:46:36.824373 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3824b24-20c5-42db-8d8f-851c319ee837 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:46:36.833831 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3824b24-20c5-42db-8d8f-851c319ee837 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:46:36.838579 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6b05d6-2227-4911-acf8-1b9d7bf813d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:46:36.839249 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6b05d6-2227-4911-acf8-1b9d7bf813d7 admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:46:36.839561 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6b05d6-2227-4911-acf8-1b9d7bf813d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:46:36.839831 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6b05d6-2227-4911-acf8-1b9d7bf813d7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:46:36.840460 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6b05d6-2227-4911-acf8-1b9d7bf813d7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:46:36.842401 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6b05d6-2227-4911-acf8-1b9d7bf813d7 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:46:36.859045 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:46:36.942806 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1293/2571] 10.222.0.30 () {60 vars in 1153 bytes} [Tue Sep 6 04:46:36 2022] POST /identity/v3/projects => generated 367 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 04:46:36.946807 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e0dc9c-f6d8-4f9d-87a2-6ea6df528376 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:46:36.957958 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e0dc9c-f6d8-4f9d-87a2-6ea6df528376 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:46:36.975077 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:46:37.016214 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d76681-7d75-47ca-9612-1e3d520718c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:46:37.016758 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d76681-7d75-47ca-9612-1e3d520718c7 admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:46:37.017074 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d76681-7d75-47ca-9612-1e3d520718c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:46:37.017387 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d76681-7d75-47ca-9612-1e3d520718c7 admin admin] PATH_INFO: `/v3/users` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:46:37.018042 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d76681-7d75-47ca-9612-1e3d520718c7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:46:37.020046 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d76681-7d75-47ca-9612-1e3d520718c7 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:46:37.096001 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1279/2572] 10.222.0.30 () {60 vars in 1144 bytes} [Tue Sep 6 04:46:36 2022] POST /identity/v3/users => generated 284 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 04:46:37.100250 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6b05d6-2227-4911-acf8-1b9d7bf813d7 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:46:37.111565 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6b05d6-2227-4911-acf8-1b9d7bf813d7 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:46:37.116425 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198ed294-f62c-4966-a65b-7603eead4ec4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:46:37.117051 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198ed294-f62c-4966-a65b-7603eead4ec4 admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:46:37.117357 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198ed294-f62c-4966-a65b-7603eead4ec4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:46:37.127881 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198ed294-f62c-4966-a65b-7603eead4ec4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:46:37.127881 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-198ed294-f62c-4966-a65b-7603eead4ec4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:46:37.127881 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-198ed294-f62c-4966-a65b-7603eead4ec4 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:46:37.134284 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1294/2573] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Sep 6 04:46:37 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 04:46:37.136252 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d76681-7d75-47ca-9612-1e3d520718c7 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:46:37.146833 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d76681-7d75-47ca-9612-1e3d520718c7 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:46:37.151859 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4d3f14-7a85-4210-9ec3-0e3ddc87acfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:46:37.156917 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4d3f14-7a85-4210-9ec3-0e3ddc87acfe admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:46:37.156917 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4d3f14-7a85-4210-9ec3-0e3ddc87acfe admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:46:37.156917 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4d3f14-7a85-4210-9ec3-0e3ddc87acfe admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:46:37.156917 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4d3f14-7a85-4210-9ec3-0e3ddc87acfe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:46:37.156917 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4d3f14-7a85-4210-9ec3-0e3ddc87acfe admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:46:37.177348 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1280/2574] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Sep 6 04:46:37 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 06 04:46:37.179953 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198ed294-f62c-4966-a65b-7603eead4ec4 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:46:37.192281 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198ed294-f62c-4966-a65b-7603eead4ec4 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:46:37.205945 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3fc5bf9-df56-4dc1-8eca-3117e709695f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:46:37.206634 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fc5bf9-df56-4dc1-8eca-3117e709695f admin admin] REQUEST_METHOD: `PUT` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:46:37.206912 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fc5bf9-df56-4dc1-8eca-3117e709695f admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:46:37.207184 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fc5bf9-df56-4dc1-8eca-3117e709695f admin admin] PATH_INFO: `/v3/projects/7ff01d886c41405f8ebdac1e5ea2b6b4/users/842b099da1ae4868be55dd53a9153e88/roles/29607b2de39f40d68bb6ceef64b8a732` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:46:37.225763 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3fc5bf9-df56-4dc1-8eca-3117e709695f admin admin] RBAC: Authorizing `identity:create_grant(project_id=7ff01d886c41405f8ebdac1e5ea2b6b4, user_id=842b099da1ae4868be55dd53a9153e88, role_id=29607b2de39f40d68bb6ceef64b8a732)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:46:37.228959 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3fc5bf9-df56-4dc1-8eca-3117e709695f admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:46:37.284960 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1295/2575] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Sep 6 04:46:37 2022] PUT /identity/v3/projects/7ff01d886c41405f8ebdac1e5ea2b6b4/users/842b099da1ae4868be55dd53a9153e88/roles/29607b2de39f40d68bb6ceef64b8a732 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 04:46:37.295114 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f81e0f-703e-4822-9cb6-3a2bb9125ff1 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:46:37.295467 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f81e0f-703e-4822-9cb6-3a2bb9125ff1 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:46:37.295811 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f81e0f-703e-4822-9cb6-3a2bb9125ff1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:46:37.327291 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-78f81e0f-703e-4822-9cb6-3a2bb9125ff1 None None] MFA Rules not processed for user `842b099da1ae4868be55dd53a9153e88`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:46:37.329104 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-78f81e0f-703e-4822-9cb6-3a2bb9125ff1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:46:37.400864 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1281/2576] 10.222.0.30 () {58 vars in 958 bytes} [Tue Sep 6 04:46:37 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 04:46:37.413495 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3fc5bf9-df56-4dc1-8eca-3117e709695f admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:46:37.415229 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-c3fc5bf9-df56-4dc1-8eca-3117e709695f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:46:37.498079 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3fc5bf9-df56-4dc1-8eca-3117e709695f admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:46:37.502358 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea59ac1-73e4-4f60-ab13-9d0b6ec87a67 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae5c31fb1d0342c0937bbc83a415113d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b707733ce65c42edbc1fd95ad0a1c06c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:46:37.502970 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea59ac1-73e4-4f60-ab13-9d0b6ec87a67 service glance] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:46:37.503249 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea59ac1-73e4-4f60-ab13-9d0b6ec87a67 service glance] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:46:37.503520 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea59ac1-73e4-4f60-ab13-9d0b6ec87a67 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:46:37.511864 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea59ac1-73e4-4f60-ab13-9d0b6ec87a67 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:46:37.513289 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea59ac1-73e4-4f60-ab13-9d0b6ec87a67 service glance] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:46:37.522978 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1296/2577] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Sep 6 04:46:37 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 04:46:47.398060 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=57,keystone:INSERT=3 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:46:47.518560 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=55,keystone:INSERT=2 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:46:53.567240 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4f7f54-5689-4d00-96c8-7efd3bda4041 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:46:53.567709 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4f7f54-5689-4d00-96c8-7efd3bda4041 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:46:53.567812 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4f7f54-5689-4d00-96c8-7efd3bda4041 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:46:53.587067 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-da4f7f54-5689-4d00-96c8-7efd3bda4041 None None] MFA Rules not processed for user `da5fed17101a4935bcc38b72caf9d263`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:46:53.588273 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-da4f7f54-5689-4d00-96c8-7efd3bda4041 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:46:53.641625 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1282/2578] 10.222.0.30 () {58 vars in 967 bytes} [Tue Sep 6 04:46:53 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 04:46:53.654494 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea59ac1-73e4-4f60-ab13-9d0b6ec87a67 service glance] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:46:53.656144 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-fea59ac1-73e4-4f60-ab13-9d0b6ec87a67 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:46:53.727402 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea59ac1-73e4-4f60-ab13-9d0b6ec87a67 service glance] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:46:53.732520 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acaa3383-dd65-4e9e-b507-716a7be10e9e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a676ec7f92a144edbeca8e6029ba9ee6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b707733ce65c42edbc1fd95ad0a1c06c', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:46:53.733164 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acaa3383-dd65-4e9e-b507-716a7be10e9e service swift] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:46:53.733478 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acaa3383-dd65-4e9e-b507-716a7be10e9e service swift] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:46:53.733755 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acaa3383-dd65-4e9e-b507-716a7be10e9e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:46:53.742291 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acaa3383-dd65-4e9e-b507-716a7be10e9e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:46:53.743887 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acaa3383-dd65-4e9e-b507-716a7be10e9e service swift] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:46:53.752148 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1297/2579] 10.222.0.30 () {60 vars in 1384 bytes} [Tue Sep 6 04:46:53 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 06 04:47:03.635165 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=25 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:47:03.748747 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=32 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:47:10.586344 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4f7f54-5689-4d00-96c8-7efd3bda4041 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:47:10.587824 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-da4f7f54-5689-4d00-96c8-7efd3bda4041 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:47:10.671793 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4f7f54-5689-4d00-96c8-7efd3bda4041 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:47:10.678382 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b5ad8f-f33d-4343-b542-da54cc717e6b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9db138b77afa4b068244b8ef715404fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b707733ce65c42edbc1fd95ad0a1c06c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:47:10.679016 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b5ad8f-f33d-4343-b542-da54cc717e6b service nova] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:47:10.679295 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b5ad8f-f33d-4343-b542-da54cc717e6b service nova] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:47:10.679570 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b5ad8f-f33d-4343-b542-da54cc717e6b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:47:10.681326 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-01b5ad8f-f33d-4343-b542-da54cc717e6b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:47:10.695407 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b5ad8f-f33d-4343-b542-da54cc717e6b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:47:10.697241 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b5ad8f-f33d-4343-b542-da54cc717e6b service nova] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:47:10.710690 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1283/2580] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Sep 6 04:47:10 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 04:47:11.790931 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acaa3383-dd65-4e9e-b507-716a7be10e9e service swift] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:47:11.793056 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-acaa3383-dd65-4e9e-b507-716a7be10e9e service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:47:11.812810 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acaa3383-dd65-4e9e-b507-716a7be10e9e service swift] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:47:11.819452 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9936278-21dc-4a14-a0b7-a282aebe3328 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9db138b77afa4b068244b8ef715404fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b707733ce65c42edbc1fd95ad0a1c06c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:47:11.820106 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9936278-21dc-4a14-a0b7-a282aebe3328 service nova] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:47:11.820390 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9936278-21dc-4a14-a0b7-a282aebe3328 service nova] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:47:11.820666 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9936278-21dc-4a14-a0b7-a282aebe3328 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:47:11.822727 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-b9936278-21dc-4a14-a0b7-a282aebe3328 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:47:11.835517 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9936278-21dc-4a14-a0b7-a282aebe3328 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:47:11.837110 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9936278-21dc-4a14-a0b7-a282aebe3328 service nova] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:47:11.848368 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1298/2581] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Sep 6 04:47:11 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 04:47:18.565709 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b5ad8f-f33d-4343-b542-da54cc717e6b service nova] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:47:18.577498 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b5ad8f-f33d-4343-b542-da54cc717e6b service nova] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:47:18.583449 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7aa73f-26cf-4fe8-ad47-906cd0fac132 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9db138b77afa4b068244b8ef715404fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b707733ce65c42edbc1fd95ad0a1c06c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:47:18.584096 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7aa73f-26cf-4fe8-ad47-906cd0fac132 service nova] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:47:18.584384 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7aa73f-26cf-4fe8-ad47-906cd0fac132 service nova] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:47:18.584631 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7aa73f-26cf-4fe8-ad47-906cd0fac132 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:47:18.586341 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-0c7aa73f-26cf-4fe8-ad47-906cd0fac132 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:47:18.664047 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c7aa73f-26cf-4fe8-ad47-906cd0fac132 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:47:18.667592 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c7aa73f-26cf-4fe8-ad47-906cd0fac132 service nova] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:47:18.684589 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1284/2582] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Sep 6 04:47:18 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 04:47:21.843751 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=6 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:47:27.770675 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5946d92e-8644-4e18-9c45-c76a13de5c6c None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:47:27.771244 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5946d92e-8644-4e18-9c45-c76a13de5c6c None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:47:27.771244 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5946d92e-8644-4e18-9c45-c76a13de5c6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:47:27.801560 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-5946d92e-8644-4e18-9c45-c76a13de5c6c None None] MFA Rules not processed for user `da5fed17101a4935bcc38b72caf9d263`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:47:27.802942 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-5946d92e-8644-4e18-9c45-c76a13de5c6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:47:27.810366 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1299/2583] 10.222.0.30 () {58 vars in 967 bytes} [Tue Sep 6 04:47:27 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 04:47:27.822619 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7aa73f-26cf-4fe8-ad47-906cd0fac132 service nova] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:47:27.834049 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7aa73f-26cf-4fe8-ad47-906cd0fac132 service nova] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:47:27.839839 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef07747e-6bfd-4ca1-b1fa-616a622fa094 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a676ec7f92a144edbeca8e6029ba9ee6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b707733ce65c42edbc1fd95ad0a1c06c', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:47:27.840497 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef07747e-6bfd-4ca1-b1fa-616a622fa094 service swift] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:47:27.840774 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef07747e-6bfd-4ca1-b1fa-616a622fa094 service swift] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:47:27.841043 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef07747e-6bfd-4ca1-b1fa-616a622fa094 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:47:27.849170 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef07747e-6bfd-4ca1-b1fa-616a622fa094 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:47:27.850808 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef07747e-6bfd-4ca1-b1fa-616a622fa094 service swift] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:47:27.859668 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1285/2584] 10.222.0.30 () {60 vars in 1384 bytes} [Tue Sep 6 04:47:27 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 06 04:47:32.183087 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5946d92e-8644-4e18-9c45-c76a13de5c6c None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:47:32.184802 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-5946d92e-8644-4e18-9c45-c76a13de5c6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:47:32.266576 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5946d92e-8644-4e18-9c45-c76a13de5c6c None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:47:32.271650 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c074395d-235d-420b-94f2-47a7fd713107 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d4f2e4dbd41441d84f03ddb30191e7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b707733ce65c42edbc1fd95ad0a1c06c', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:47:32.272261 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c074395d-235d-420b-94f2-47a7fd713107 service placement] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:47:32.272539 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c074395d-235d-420b-94f2-47a7fd713107 service placement] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:47:32.272813 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c074395d-235d-420b-94f2-47a7fd713107 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:47:32.275404 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-c074395d-235d-420b-94f2-47a7fd713107 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:47:32.286887 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c074395d-235d-420b-94f2-47a7fd713107 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:47:32.288470 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c074395d-235d-420b-94f2-47a7fd713107 service placement] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:47:32.300139 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1300/2585] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Sep 6 04:47:32 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 04:47:32.576075 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef07747e-6bfd-4ca1-b1fa-616a622fa094 service swift] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:47:32.622234 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:47:32.640028 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef07747e-6bfd-4ca1-b1fa-616a622fa094 service swift] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:47:32.644632 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a26ea4-6b56-47e2-879f-e2138ee5c00b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:47:32.645278 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a26ea4-6b56-47e2-879f-e2138ee5c00b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:47:32.645588 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a26ea4-6b56-47e2-879f-e2138ee5c00b admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:47:32.645863 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a26ea4-6b56-47e2-879f-e2138ee5c00b admin admin] PATH_INFO: `/v3/users/842b099da1ae4868be55dd53a9153e88` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:47:32.647000 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a26ea4-6b56-47e2-879f-e2138ee5c00b admin admin] RBAC: Authorizing `identity:delete_user(user_id=842b099da1ae4868be55dd53a9153e88)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:47:32.649933 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a26ea4-6b56-47e2-879f-e2138ee5c00b admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:47:32.805902 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-f0a26ea4-6b56-47e2-879f-e2138ee5c00b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '842b099da1ae4868be55dd53a9153e88'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:47:32.848984 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-f0a26ea4-6b56-47e2-879f-e2138ee5c00b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '842b099da1ae4868be55dd53a9153e88'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:47:32.849760 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-f0a26ea4-6b56-47e2-879f-e2138ee5c00b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '842b099da1ae4868be55dd53a9153e88'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:47:32.857299 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-f0a26ea4-6b56-47e2-879f-e2138ee5c00b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '842b099da1ae4868be55dd53a9153e88'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:47:32.870403 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-f0a26ea4-6b56-47e2-879f-e2138ee5c00b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '842b099da1ae4868be55dd53a9153e88'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:47:32.876322 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1286/2586] 10.222.0.30 () {58 vars in 1224 bytes} [Tue Sep 6 04:47:32 2022] DELETE /identity/v3/users/842b099da1ae4868be55dd53a9153e88 => generated 0 bytes in 301 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 04:47:32.880756 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bff9394-c1c3-4669-bde5-6d439dfe0c3d None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:47:32.881048 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bff9394-c1c3-4669-bde5-6d439dfe0c3d None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:47:32.881350 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bff9394-c1c3-4669-bde5-6d439dfe0c3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:47:32.901363 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-5bff9394-c1c3-4669-bde5-6d439dfe0c3d None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:47:32.902585 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-5bff9394-c1c3-4669-bde5-6d439dfe0c3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:47:32.962837 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1301/2587] 10.222.0.30 () {58 vars in 958 bytes} [Tue Sep 6 04:47:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 04:47:32.972535 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a26ea4-6b56-47e2-879f-e2138ee5c00b admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:47:32.973957 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-f0a26ea4-6b56-47e2-879f-e2138ee5c00b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:47:33.032469 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a26ea4-6b56-47e2-879f-e2138ee5c00b admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:47:33.036132 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed624812-0ad7-4244-a446-0216a112edab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90492440f6a6471e8392361c098428aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b707733ce65c42edbc1fd95ad0a1c06c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:47:33.036752 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed624812-0ad7-4244-a446-0216a112edab service neutron] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:47:33.037026 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed624812-0ad7-4244-a446-0216a112edab service neutron] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:47:33.037315 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed624812-0ad7-4244-a446-0216a112edab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:47:33.046628 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed624812-0ad7-4244-a446-0216a112edab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:47:33.048081 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed624812-0ad7-4244-a446-0216a112edab service neutron] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:47:33.058378 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1287/2588] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Sep 6 04:47:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 04:47:33.433194 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bff9394-c1c3-4669-bde5-6d439dfe0c3d None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:47:33.434995 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-5bff9394-c1c3-4669-bde5-6d439dfe0c3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:47:33.448792 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bff9394-c1c3-4669-bde5-6d439dfe0c3d None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:47:33.453653 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80977750-cd5d-4d23-ad5e-d68f1acc86a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:47:33.454545 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80977750-cd5d-4d23-ad5e-d68f1acc86a5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:47:33.455011 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80977750-cd5d-4d23-ad5e-d68f1acc86a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:47:33.455530 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80977750-cd5d-4d23-ad5e-d68f1acc86a5 admin admin] PATH_INFO: `/v3/projects/7ff01d886c41405f8ebdac1e5ea2b6b4` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:47:33.456991 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80977750-cd5d-4d23-ad5e-d68f1acc86a5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7ff01d886c41405f8ebdac1e5ea2b6b4)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:47:33.459351 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80977750-cd5d-4d23-ad5e-d68f1acc86a5 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:47:33.573707 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-80977750-cd5d-4d23-ad5e-d68f1acc86a5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7ff01d886c41405f8ebdac1e5ea2b6b4', 'request_id': 'req-80977750-cd5d-4d23-ad5e-d68f1acc86a5'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:47:33.579965 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-80977750-cd5d-4d23-ad5e-d68f1acc86a5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7ff01d886c41405f8ebdac1e5ea2b6b4', 'request_id': 'req-80977750-cd5d-4d23-ad5e-d68f1acc86a5'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:47:33.666142 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-80977750-cd5d-4d23-ad5e-d68f1acc86a5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7ff01d886c41405f8ebdac1e5ea2b6b4', 'request_id': 'req-80977750-cd5d-4d23-ad5e-d68f1acc86a5'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:47:33.679261 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-80977750-cd5d-4d23-ad5e-d68f1acc86a5 admin admin] The token cache is being invalidate because project 7ff01d886c41405f8ebdac1e5ea2b6b4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81698) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 04:47:33.680071 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-80977750-cd5d-4d23-ad5e-d68f1acc86a5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:47:33.680795 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-80977750-cd5d-4d23-ad5e-d68f1acc86a5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:47:33.682618 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1302/2589] 10.222.0.30 () {58 vars in 1233 bytes} [Tue Sep 6 04:47:33 2022] DELETE /identity/v3/projects/7ff01d886c41405f8ebdac1e5ea2b6b4 => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 04:47:33.990020 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed624812-0ad7-4244-a446-0216a112edab service neutron] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:47:33.999231 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed624812-0ad7-4244-a446-0216a112edab service neutron] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:47:34.050829 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a5185c-18bc-4ce9-855b-7022934bbd36 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90492440f6a6471e8392361c098428aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b707733ce65c42edbc1fd95ad0a1c06c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:47:34.052943 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a5185c-18bc-4ce9-855b-7022934bbd36 service neutron] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:47:34.052943 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a5185c-18bc-4ce9-855b-7022934bbd36 service neutron] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:47:34.052943 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a5185c-18bc-4ce9-855b-7022934bbd36 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:47:34.054213 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-82a5185c-18bc-4ce9-855b-7022934bbd36 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:47:34.074364 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:47:34.122418 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a5185c-18bc-4ce9-855b-7022934bbd36 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:47:34.123856 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a5185c-18bc-4ce9-855b-7022934bbd36 service neutron] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:47:34.134486 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1288/2590] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Sep 6 04:47:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 04:47:36.442817 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80977750-cd5d-4d23-ad5e-d68f1acc86a5 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:47:36.444269 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-80977750-cd5d-4d23-ad5e-d68f1acc86a5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:47:36.457779 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80977750-cd5d-4d23-ad5e-d68f1acc86a5 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:47:36.463145 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c552cdb-f746-4d9c-83a2-dd05d6bb6af7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:47:36.464041 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c552cdb-f746-4d9c-83a2-dd05d6bb6af7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:47:36.464041 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c552cdb-f746-4d9c-83a2-dd05d6bb6af7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:47:36.464296 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c552cdb-f746-4d9c-83a2-dd05d6bb6af7 admin admin] PATH_INFO: `/v3/users/9f07caa54c8f47778bb984bfd9800dc0` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:47:36.465447 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c552cdb-f746-4d9c-83a2-dd05d6bb6af7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9f07caa54c8f47778bb984bfd9800dc0)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:47:36.467375 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c552cdb-f746-4d9c-83a2-dd05d6bb6af7 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:47:36.469016 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:47:36.657997 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-0c552cdb-f746-4d9c-83a2-dd05d6bb6af7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9f07caa54c8f47778bb984bfd9800dc0'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:47:36.721029 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-0c552cdb-f746-4d9c-83a2-dd05d6bb6af7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9f07caa54c8f47778bb984bfd9800dc0'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:47:36.721672 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-0c552cdb-f746-4d9c-83a2-dd05d6bb6af7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9f07caa54c8f47778bb984bfd9800dc0'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:47:36.734696 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-0c552cdb-f746-4d9c-83a2-dd05d6bb6af7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9f07caa54c8f47778bb984bfd9800dc0'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:47:36.741523 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-0c552cdb-f746-4d9c-83a2-dd05d6bb6af7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9f07caa54c8f47778bb984bfd9800dc0'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:47:36.748126 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1303/2591] 10.222.0.30 () {58 vars in 1224 bytes} [Tue Sep 6 04:47:36 2022] DELETE /identity/v3/users/9f07caa54c8f47778bb984bfd9800dc0 => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 04:47:37.010403 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a5185c-18bc-4ce9-855b-7022934bbd36 service neutron] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:47:37.024116 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a5185c-18bc-4ce9-855b-7022934bbd36 service neutron] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:47:37.201780 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a0fd52-1513-412b-9e11-f8b7c42102fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:47:37.202338 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a0fd52-1513-412b-9e11-f8b7c42102fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:47:37.202619 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a0fd52-1513-412b-9e11-f8b7c42102fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:47:37.202889 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a0fd52-1513-412b-9e11-f8b7c42102fc admin admin] PATH_INFO: `/v3/projects/623b9f3be0f24e2ea1caea0798b30704` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:47:37.204047 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a0fd52-1513-412b-9e11-f8b7c42102fc admin admin] RBAC: Authorizing `identity:delete_project(project_id=623b9f3be0f24e2ea1caea0798b30704)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:47:37.205982 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a0fd52-1513-412b-9e11-f8b7c42102fc admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:47:37.302229 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-b7a0fd52-1513-412b-9e11-f8b7c42102fc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '623b9f3be0f24e2ea1caea0798b30704', 'request_id': 'req-b7a0fd52-1513-412b-9e11-f8b7c42102fc'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:47:37.362658 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-b7a0fd52-1513-412b-9e11-f8b7c42102fc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '623b9f3be0f24e2ea1caea0798b30704', 'request_id': 'req-b7a0fd52-1513-412b-9e11-f8b7c42102fc'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:47:37.371197 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-b7a0fd52-1513-412b-9e11-f8b7c42102fc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '623b9f3be0f24e2ea1caea0798b30704', 'request_id': 'req-b7a0fd52-1513-412b-9e11-f8b7c42102fc'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:47:37.387509 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-b7a0fd52-1513-412b-9e11-f8b7c42102fc admin admin] The token cache is being invalidate because project 623b9f3be0f24e2ea1caea0798b30704 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81697) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 04:47:37.387801 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-b7a0fd52-1513-412b-9e11-f8b7c42102fc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:47:37.397037 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-b7a0fd52-1513-412b-9e11-f8b7c42102fc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:47:37.397037 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1289/2592] 10.222.0.30 () {58 vars in 1233 bytes} [Tue Sep 6 04:47:37 2022] DELETE /identity/v3/projects/623b9f3be0f24e2ea1caea0798b30704 => generated 0 bytes in 380 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 04:47:37.397272 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c552cdb-f746-4d9c-83a2-dd05d6bb6af7 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:47:37.397272 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-0c552cdb-f746-4d9c-83a2-dd05d6bb6af7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:47:37.410118 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c552cdb-f746-4d9c-83a2-dd05d6bb6af7 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:47:37.472869 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74fa2d78-8140-4566-930c-06e6354b36ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:47:37.473576 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fa2d78-8140-4566-930c-06e6354b36ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:47:37.473735 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fa2d78-8140-4566-930c-06e6354b36ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:47:37.474012 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fa2d78-8140-4566-930c-06e6354b36ff admin admin] PATH_INFO: `/v3/users/dae02250da564a23bc090baffe73a30f` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:47:37.475165 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74fa2d78-8140-4566-930c-06e6354b36ff admin admin] RBAC: Authorizing `identity:delete_user(user_id=dae02250da564a23bc090baffe73a30f)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:47:37.478931 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74fa2d78-8140-4566-930c-06e6354b36ff admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:47:37.650258 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-74fa2d78-8140-4566-930c-06e6354b36ff admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dae02250da564a23bc090baffe73a30f'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:47:37.734085 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-74fa2d78-8140-4566-930c-06e6354b36ff admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dae02250da564a23bc090baffe73a30f'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:47:37.734751 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-74fa2d78-8140-4566-930c-06e6354b36ff admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dae02250da564a23bc090baffe73a30f'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:47:37.768262 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-74fa2d78-8140-4566-930c-06e6354b36ff admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dae02250da564a23bc090baffe73a30f'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:47:37.776538 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-74fa2d78-8140-4566-930c-06e6354b36ff admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dae02250da564a23bc090baffe73a30f'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:47:37.778172 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:47:37.783420 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1304/2593] 10.222.0.30 () {58 vars in 1224 bytes} [Tue Sep 6 04:47:37 2022] DELETE /identity/v3/users/dae02250da564a23bc090baffe73a30f => generated 0 bytes in 390 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 04:47:38.020698 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a0fd52-1513-412b-9e11-f8b7c42102fc admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:47:38.024368 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-b7a0fd52-1513-412b-9e11-f8b7c42102fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:47:38.035808 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a0fd52-1513-412b-9e11-f8b7c42102fc admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:47:38.078813 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:47:38.093697 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae93e65-6f57-4b86-9252-d9fffdbd9a62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:47:38.094323 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae93e65-6f57-4b86-9252-d9fffdbd9a62 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:47:38.094596 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae93e65-6f57-4b86-9252-d9fffdbd9a62 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:47:38.094864 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae93e65-6f57-4b86-9252-d9fffdbd9a62 admin admin] PATH_INFO: `/v3/projects/e5f9492622a543b4808f5816cb2afdf9` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:47:38.095958 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae93e65-6f57-4b86-9252-d9fffdbd9a62 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e5f9492622a543b4808f5816cb2afdf9)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:47:38.097866 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae93e65-6f57-4b86-9252-d9fffdbd9a62 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:47:38.178376 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-aae93e65-6f57-4b86-9252-d9fffdbd9a62 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e5f9492622a543b4808f5816cb2afdf9', 'request_id': 'req-aae93e65-6f57-4b86-9252-d9fffdbd9a62'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:47:38.224738 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-aae93e65-6f57-4b86-9252-d9fffdbd9a62 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e5f9492622a543b4808f5816cb2afdf9', 'request_id': 'req-aae93e65-6f57-4b86-9252-d9fffdbd9a62'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:47:38.231090 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-aae93e65-6f57-4b86-9252-d9fffdbd9a62 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e5f9492622a543b4808f5816cb2afdf9', 'request_id': 'req-aae93e65-6f57-4b86-9252-d9fffdbd9a62'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:47:38.241681 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-aae93e65-6f57-4b86-9252-d9fffdbd9a62 admin admin] The token cache is being invalidate because project e5f9492622a543b4808f5816cb2afdf9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81697) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 04:47:38.241961 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-aae93e65-6f57-4b86-9252-d9fffdbd9a62 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:47:38.242586 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-aae93e65-6f57-4b86-9252-d9fffdbd9a62 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:47:38.244806 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1290/2594] 10.222.0.30 () {58 vars in 1233 bytes} [Tue Sep 6 04:47:38 2022] DELETE /identity/v3/projects/e5f9492622a543b4808f5816cb2afdf9 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 04:47:47.892169 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=1 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:47:48.238593 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=31,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:47:48.800248 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74fa2d78-8140-4566-930c-06e6354b36ff admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:47:48.802052 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-74fa2d78-8140-4566-930c-06e6354b36ff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:47:48.858810 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74fa2d78-8140-4566-930c-06e6354b36ff admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:47:48.905007 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e0c3b8-0c50-4a55-b848-ccb1d463f30f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90492440f6a6471e8392361c098428aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b707733ce65c42edbc1fd95ad0a1c06c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:47:48.905902 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e0c3b8-0c50-4a55-b848-ccb1d463f30f service neutron] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:47:48.906152 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e0c3b8-0c50-4a55-b848-ccb1d463f30f service neutron] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:47:48.906422 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e0c3b8-0c50-4a55-b848-ccb1d463f30f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:47:48.908115 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-04e0c3b8-0c50-4a55-b848-ccb1d463f30f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:47:48.967838 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e0c3b8-0c50-4a55-b848-ccb1d463f30f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:47:48.969260 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e0c3b8-0c50-4a55-b848-ccb1d463f30f service neutron] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:47:48.980230 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1305/2595] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Sep 6 04:47:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 04:47:52.500274 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae93e65-6f57-4b86-9252-d9fffdbd9a62 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:47:52.502081 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-aae93e65-6f57-4b86-9252-d9fffdbd9a62 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:47:52.515296 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae93e65-6f57-4b86-9252-d9fffdbd9a62 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:47:52.519884 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ca2c9e-72df-49dc-a0a4-9824289e6fb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:47:52.520521 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ca2c9e-72df-49dc-a0a4-9824289e6fb4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:47:52.520851 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ca2c9e-72df-49dc-a0a4-9824289e6fb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:47:52.521123 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ca2c9e-72df-49dc-a0a4-9824289e6fb4 admin admin] PATH_INFO: `/v3/users/5a0294ea40914bf290902307a787642d` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:47:52.522325 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ca2c9e-72df-49dc-a0a4-9824289e6fb4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5a0294ea40914bf290902307a787642d)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:47:52.524239 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ca2c9e-72df-49dc-a0a4-9824289e6fb4 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:47:52.678753 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-82ca2c9e-72df-49dc-a0a4-9824289e6fb4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5a0294ea40914bf290902307a787642d'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:47:52.733269 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-82ca2c9e-72df-49dc-a0a4-9824289e6fb4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5a0294ea40914bf290902307a787642d'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:47:52.733917 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-82ca2c9e-72df-49dc-a0a4-9824289e6fb4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5a0294ea40914bf290902307a787642d'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:47:52.741257 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-82ca2c9e-72df-49dc-a0a4-9824289e6fb4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5a0294ea40914bf290902307a787642d'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:47:52.753459 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-82ca2c9e-72df-49dc-a0a4-9824289e6fb4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5a0294ea40914bf290902307a787642d'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:47:52.759990 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1291/2596] 10.222.0.30 () {58 vars in 1224 bytes} [Tue Sep 6 04:47:52 2022] DELETE /identity/v3/users/5a0294ea40914bf290902307a787642d => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 04:47:53.001884 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e0c3b8-0c50-4a55-b848-ccb1d463f30f service neutron] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:47:53.016086 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e0c3b8-0c50-4a55-b848-ccb1d463f30f service neutron] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:47:53.076194 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7ba83f-83ee-48ca-846c-b2f27fd0d733 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:47:53.076862 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7ba83f-83ee-48ca-846c-b2f27fd0d733 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:47:53.077143 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7ba83f-83ee-48ca-846c-b2f27fd0d733 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:47:53.077448 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7ba83f-83ee-48ca-846c-b2f27fd0d733 admin admin] PATH_INFO: `/v3/projects/3d04ba57221f42d684873d624dbfd283` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:47:53.078583 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7ba83f-83ee-48ca-846c-b2f27fd0d733 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3d04ba57221f42d684873d624dbfd283)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:47:53.080496 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7ba83f-83ee-48ca-846c-b2f27fd0d733 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:47:53.089471 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:47:53.200926 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-6f7ba83f-83ee-48ca-846c-b2f27fd0d733 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3d04ba57221f42d684873d624dbfd283', 'request_id': 'req-6f7ba83f-83ee-48ca-846c-b2f27fd0d733'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:47:53.206545 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-6f7ba83f-83ee-48ca-846c-b2f27fd0d733 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3d04ba57221f42d684873d624dbfd283', 'request_id': 'req-6f7ba83f-83ee-48ca-846c-b2f27fd0d733'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:47:53.292140 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-6f7ba83f-83ee-48ca-846c-b2f27fd0d733 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3d04ba57221f42d684873d624dbfd283', 'request_id': 'req-6f7ba83f-83ee-48ca-846c-b2f27fd0d733'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:47:53.303453 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-6f7ba83f-83ee-48ca-846c-b2f27fd0d733 admin admin] The token cache is being invalidate because project 3d04ba57221f42d684873d624dbfd283 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81698) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 04:47:53.308342 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-6f7ba83f-83ee-48ca-846c-b2f27fd0d733 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:47:53.308342 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-6f7ba83f-83ee-48ca-846c-b2f27fd0d733 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:47:53.308342 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1306/2597] 10.222.0.30 () {58 vars in 1233 bytes} [Tue Sep 6 04:47:53 2022] DELETE /identity/v3/projects/3d04ba57221f42d684873d624dbfd283 => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 04:47:53.311410 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f186d5-c6f0-4824-b647-b56cfa5f2e4c None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:47:53.311817 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f186d5-c6f0-4824-b647-b56cfa5f2e4c None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:47:53.312106 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f186d5-c6f0-4824-b647-b56cfa5f2e4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:47:53.331768 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-02f186d5-c6f0-4824-b647-b56cfa5f2e4c None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:47:53.333256 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-02f186d5-c6f0-4824-b647-b56cfa5f2e4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:47:53.395245 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1292/2598] 10.222.0.30 () {58 vars in 958 bytes} [Tue Sep 6 04:47:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 04:47:53.403226 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61eb232-e8f9-417e-adea-4cb29ccd8774 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:47:53.403511 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61eb232-e8f9-417e-adea-4cb29ccd8774 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:47:53.403785 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61eb232-e8f9-417e-adea-4cb29ccd8774 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:47:53.424858 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-b61eb232-e8f9-417e-adea-4cb29ccd8774 None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:47:53.426093 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-b61eb232-e8f9-417e-adea-4cb29ccd8774 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:47:53.434822 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1307/2599] 10.222.0.30 () {58 vars in 958 bytes} [Tue Sep 6 04:47:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 04:47:53.439185 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f186d5-c6f0-4824-b647-b56cfa5f2e4c None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:47:53.449489 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f186d5-c6f0-4824-b647-b56cfa5f2e4c None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:47:53.454092 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830a4100-2519-4950-af85-281aac4df14f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:47:53.454717 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830a4100-2519-4950-af85-281aac4df14f admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:47:53.454998 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830a4100-2519-4950-af85-281aac4df14f admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:47:53.455270 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830a4100-2519-4950-af85-281aac4df14f admin admin] PATH_INFO: `/v3/domains` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:47:53.455915 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-830a4100-2519-4950-af85-281aac4df14f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:47:53.456217 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-830a4100-2519-4950-af85-281aac4df14f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:47:53.457844 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-830a4100-2519-4950-af85-281aac4df14f admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:47:53.469494 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1293/2600] 10.222.0.30 () {58 vars in 1153 bytes} [Tue Sep 6 04:47:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 04:47:53.473728 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61eb232-e8f9-417e-adea-4cb29ccd8774 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:47:53.483407 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61eb232-e8f9-417e-adea-4cb29ccd8774 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:47:53.488304 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea8dbbf-0a58-4cee-9f1b-980e2da1f462 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:47:53.488903 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea8dbbf-0a58-4cee-9f1b-980e2da1f462 admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:47:53.489179 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea8dbbf-0a58-4cee-9f1b-980e2da1f462 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:47:53.489485 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea8dbbf-0a58-4cee-9f1b-980e2da1f462 admin admin] PATH_INFO: `/v3/projects` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:47:53.490128 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea8dbbf-0a58-4cee-9f1b-980e2da1f462 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:47:53.492051 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea8dbbf-0a58-4cee-9f1b-980e2da1f462 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:47:53.569158 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1308/2601] 10.222.0.30 () {60 vars in 1153 bytes} [Tue Sep 6 04:47:53 2022] POST /identity/v3/projects => generated 403 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 04:47:53.574098 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830a4100-2519-4950-af85-281aac4df14f admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:47:53.584102 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830a4100-2519-4950-af85-281aac4df14f admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:47:53.639914 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93700ea-a47e-4df8-be4d-d79257b67aa7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:47:53.640509 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93700ea-a47e-4df8-be4d-d79257b67aa7 admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:47:53.640787 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93700ea-a47e-4df8-be4d-d79257b67aa7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:47:53.641054 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93700ea-a47e-4df8-be4d-d79257b67aa7 admin admin] PATH_INFO: `/v3/users` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:47:53.641764 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f93700ea-a47e-4df8-be4d-d79257b67aa7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:47:53.643732 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f93700ea-a47e-4df8-be4d-d79257b67aa7 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:47:53.652345 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:47:53.777634 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1294/2602] 10.222.0.30 () {60 vars in 1144 bytes} [Tue Sep 6 04:47:53 2022] POST /identity/v3/users => generated 302 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 04:47:53.777786 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea8dbbf-0a58-4cee-9f1b-980e2da1f462 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:47:53.785755 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea8dbbf-0a58-4cee-9f1b-980e2da1f462 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:47:53.790387 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab58d0a9-2ea4-4118-9f34-56259b12dd6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:47:53.790983 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab58d0a9-2ea4-4118-9f34-56259b12dd6f admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:47:53.791269 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab58d0a9-2ea4-4118-9f34-56259b12dd6f admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:47:53.791545 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab58d0a9-2ea4-4118-9f34-56259b12dd6f admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:47:53.792186 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab58d0a9-2ea4-4118-9f34-56259b12dd6f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:47:53.793837 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab58d0a9-2ea4-4118-9f34-56259b12dd6f admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:47:53.804611 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1309/2603] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Sep 6 04:47:53 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 04:47:53.809147 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93700ea-a47e-4df8-be4d-d79257b67aa7 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:47:53.819307 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93700ea-a47e-4df8-be4d-d79257b67aa7 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:47:53.824637 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582b8c8c-af2a-466b-80ff-911ef03a897a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:47:53.825519 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582b8c8c-af2a-466b-80ff-911ef03a897a admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:47:53.826162 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582b8c8c-af2a-466b-80ff-911ef03a897a admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:47:53.826537 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582b8c8c-af2a-466b-80ff-911ef03a897a admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:47:53.827228 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-582b8c8c-af2a-466b-80ff-911ef03a897a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:47:53.828837 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-582b8c8c-af2a-466b-80ff-911ef03a897a admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:47:53.839265 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1295/2604] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Sep 6 04:47:53 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 04:47:53.843136 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab58d0a9-2ea4-4118-9f34-56259b12dd6f admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:47:53.853631 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab58d0a9-2ea4-4118-9f34-56259b12dd6f admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:47:53.858202 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f37bde-36fd-4f23-a74c-5eaca0df4ecb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:47:53.858870 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f37bde-36fd-4f23-a74c-5eaca0df4ecb admin admin] REQUEST_METHOD: `PUT` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:47:53.859150 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f37bde-36fd-4f23-a74c-5eaca0df4ecb admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:47:53.859425 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f37bde-36fd-4f23-a74c-5eaca0df4ecb admin admin] PATH_INFO: `/v3/projects/81e530a19a7c459c95377ed0ec6b8565/users/bd9830058b08458facb66090c8fafcef/roles/29607b2de39f40d68bb6ceef64b8a732` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:47:53.874186 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f37bde-36fd-4f23-a74c-5eaca0df4ecb admin admin] RBAC: Authorizing `identity:create_grant(project_id=81e530a19a7c459c95377ed0ec6b8565, user_id=bd9830058b08458facb66090c8fafcef, role_id=29607b2de39f40d68bb6ceef64b8a732)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:47:53.876817 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f37bde-36fd-4f23-a74c-5eaca0df4ecb admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:47:53.939126 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1310/2605] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Sep 6 04:47:53 2022] PUT /identity/v3/projects/81e530a19a7c459c95377ed0ec6b8565/users/bd9830058b08458facb66090c8fafcef/roles/29607b2de39f40d68bb6ceef64b8a732 => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 04:47:53.950777 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b438a5ac-a89c-42d6-8ebf-7d6b2f288684 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:47:53.951063 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b438a5ac-a89c-42d6-8ebf-7d6b2f288684 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:47:53.951338 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b438a5ac-a89c-42d6-8ebf-7d6b2f288684 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:47:53.979439 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-b438a5ac-a89c-42d6-8ebf-7d6b2f288684 None None] MFA Rules not processed for user `bd9830058b08458facb66090c8fafcef`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:47:53.980615 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-b438a5ac-a89c-42d6-8ebf-7d6b2f288684 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:47:54.043439 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1296/2606] 10.222.0.30 () {58 vars in 958 bytes} [Tue Sep 6 04:47:53 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 04:47:54.047922 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f37bde-36fd-4f23-a74c-5eaca0df4ecb admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:47:54.111397 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f37bde-36fd-4f23-a74c-5eaca0df4ecb admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:47:54.116127 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64440d14-e115-48af-a184-88a55e89e6fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:47:54.116736 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64440d14-e115-48af-a184-88a55e89e6fb admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:47:54.118062 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64440d14-e115-48af-a184-88a55e89e6fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:47:54.118062 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64440d14-e115-48af-a184-88a55e89e6fb admin admin] PATH_INFO: `/v3/projects` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:47:54.118062 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64440d14-e115-48af-a184-88a55e89e6fb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:47:54.120463 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64440d14-e115-48af-a184-88a55e89e6fb admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:47:54.196128 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1311/2607] 10.222.0.30 () {60 vars in 1153 bytes} [Tue Sep 6 04:47:54 2022] POST /identity/v3/projects => generated 403 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 04:47:54.200507 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b438a5ac-a89c-42d6-8ebf-7d6b2f288684 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:47:54.211205 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b438a5ac-a89c-42d6-8ebf-7d6b2f288684 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:47:54.266718 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c306cbd-4d8f-4e09-8c91-337e266897bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:47:54.267320 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c306cbd-4d8f-4e09-8c91-337e266897bb admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:47:54.267598 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c306cbd-4d8f-4e09-8c91-337e266897bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:47:54.267865 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c306cbd-4d8f-4e09-8c91-337e266897bb admin admin] PATH_INFO: `/v3/users` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:47:54.268484 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c306cbd-4d8f-4e09-8c91-337e266897bb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:47:54.270393 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c306cbd-4d8f-4e09-8c91-337e266897bb admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:47:54.344283 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1297/2608] 10.222.0.30 () {60 vars in 1144 bytes} [Tue Sep 6 04:47:54 2022] POST /identity/v3/users => generated 308 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 04:47:54.348272 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64440d14-e115-48af-a184-88a55e89e6fb admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:47:54.359789 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64440d14-e115-48af-a184-88a55e89e6fb admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:47:54.364509 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87cd982d-c3da-41e9-bcc2-90afb4f31760 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:47:54.365161 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cd982d-c3da-41e9-bcc2-90afb4f31760 admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:47:54.365475 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cd982d-c3da-41e9-bcc2-90afb4f31760 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:47:54.365756 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cd982d-c3da-41e9-bcc2-90afb4f31760 admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:47:54.366397 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87cd982d-c3da-41e9-bcc2-90afb4f31760 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:47:54.368019 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87cd982d-c3da-41e9-bcc2-90afb4f31760 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:47:54.380365 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1312/2609] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Sep 6 04:47:54 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 04:47:54.385524 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c306cbd-4d8f-4e09-8c91-337e266897bb admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:47:54.395516 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c306cbd-4d8f-4e09-8c91-337e266897bb admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:47:54.400775 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f089b2-7fab-4180-8e05-77e5a086b7b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:47:54.401482 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f089b2-7fab-4180-8e05-77e5a086b7b4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:47:54.401761 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f089b2-7fab-4180-8e05-77e5a086b7b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:47:54.402031 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f089b2-7fab-4180-8e05-77e5a086b7b4 admin admin] PATH_INFO: `/v3/projects/5bf8e36373d64a0c9766b5297228a40c/users/570d49fb567245f29047aa3774df08a9/roles/ea4c814713b149108e488de855b025e4` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:47:54.417188 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f089b2-7fab-4180-8e05-77e5a086b7b4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5bf8e36373d64a0c9766b5297228a40c, user_id=570d49fb567245f29047aa3774df08a9, role_id=ea4c814713b149108e488de855b025e4)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:47:54.419934 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f089b2-7fab-4180-8e05-77e5a086b7b4 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:47:54.496737 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1298/2610] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Sep 6 04:47:54 2022] PUT /identity/v3/projects/5bf8e36373d64a0c9766b5297228a40c/users/570d49fb567245f29047aa3774df08a9/roles/ea4c814713b149108e488de855b025e4 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 04:47:54.500766 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87cd982d-c3da-41e9-bcc2-90afb4f31760 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:47:54.511319 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87cd982d-c3da-41e9-bcc2-90afb4f31760 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:47:54.514933 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:47:54.566811 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f6c5b7-6a15-4321-8d4e-ae94b0cd4096 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:47:54.567620 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f6c5b7-6a15-4321-8d4e-ae94b0cd4096 admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:47:54.568077 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f6c5b7-6a15-4321-8d4e-ae94b0cd4096 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:47:54.568705 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f6c5b7-6a15-4321-8d4e-ae94b0cd4096 admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:47:54.569427 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f6c5b7-6a15-4321-8d4e-ae94b0cd4096 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:47:54.572254 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f6c5b7-6a15-4321-8d4e-ae94b0cd4096 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:47:54.581340 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1313/2611] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Sep 6 04:47:54 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 04:47:54.585679 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f089b2-7fab-4180-8e05-77e5a086b7b4 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:47:54.600989 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f089b2-7fab-4180-8e05-77e5a086b7b4 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:47:54.601146 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ff2423-9bbb-4c47-a777-afc874ae3dda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:47:54.601770 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ff2423-9bbb-4c47-a777-afc874ae3dda admin admin] REQUEST_METHOD: `PUT` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:47:54.602044 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ff2423-9bbb-4c47-a777-afc874ae3dda admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:47:54.602307 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ff2423-9bbb-4c47-a777-afc874ae3dda admin admin] PATH_INFO: `/v3/projects/5bf8e36373d64a0c9766b5297228a40c/users/570d49fb567245f29047aa3774df08a9/roles/ea4c814713b149108e488de855b025e4` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:47:54.604425 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ff2423-9bbb-4c47-a777-afc874ae3dda admin admin] RBAC: Authorizing `identity:create_grant(project_id=5bf8e36373d64a0c9766b5297228a40c, user_id=570d49fb567245f29047aa3774df08a9, role_id=ea4c814713b149108e488de855b025e4)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:47:54.607195 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ff2423-9bbb-4c47-a777-afc874ae3dda admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:47:54.617132 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1299/2612] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Sep 6 04:47:54 2022] PUT /identity/v3/projects/5bf8e36373d64a0c9766b5297228a40c/users/570d49fb567245f29047aa3774df08a9/roles/ea4c814713b149108e488de855b025e4 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 04:47:54.628082 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03465158-4039-46ce-a4ef-c6161d3583fe None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:47:54.629681 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03465158-4039-46ce-a4ef-c6161d3583fe None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:47:54.629681 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03465158-4039-46ce-a4ef-c6161d3583fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:47:54.656904 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-03465158-4039-46ce-a4ef-c6161d3583fe None None] MFA Rules not processed for user `570d49fb567245f29047aa3774df08a9`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:47:54.658159 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-03465158-4039-46ce-a4ef-c6161d3583fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:47:54.723886 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1314/2613] 10.222.0.30 () {58 vars in 958 bytes} [Tue Sep 6 04:47:54 2022] POST /identity/v3/auth/tokens => generated 3347 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 04:47:54.734804 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ff2423-9bbb-4c47-a777-afc874ae3dda admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:47:54.736481 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-25ff2423-9bbb-4c47-a777-afc874ae3dda admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:47:54.759652 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:INSERT=8,keystone:SELECT=92 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:47:54.795141 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ff2423-9bbb-4c47-a777-afc874ae3dda admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:47:54.798817 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08449e68-ec71-4f90-a89d-0122680b49b2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd09b5ab675634a2eb4b03755743f7d0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b707733ce65c42edbc1fd95ad0a1c06c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:47:54.799418 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08449e68-ec71-4f90-a89d-0122680b49b2 service cinder] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:47:54.799693 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08449e68-ec71-4f90-a89d-0122680b49b2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:47:54.799960 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08449e68-ec71-4f90-a89d-0122680b49b2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:47:54.809892 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08449e68-ec71-4f90-a89d-0122680b49b2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:47:54.811311 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08449e68-ec71-4f90-a89d-0122680b49b2 service cinder] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:47:54.820795 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1300/2614] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Sep 6 04:47:54 2022] GET /identity/v3/auth/tokens => generated 3347 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 04:47:54.980218 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03465158-4039-46ce-a4ef-c6161d3583fe None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:47:55.043176 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03465158-4039-46ce-a4ef-c6161d3583fe None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:47:55.047753 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d738eef3-711b-44ad-83b9-ebfcbba90256 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:47:55.048378 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d738eef3-711b-44ad-83b9-ebfcbba90256 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:47:55.053618 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d738eef3-711b-44ad-83b9-ebfcbba90256 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:47:55.053618 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d738eef3-711b-44ad-83b9-ebfcbba90256 admin admin] PATH_INFO: `/v3/users/bd9830058b08458facb66090c8fafcef` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:47:55.053618 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d738eef3-711b-44ad-83b9-ebfcbba90256 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bd9830058b08458facb66090c8fafcef)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:47:55.053618 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d738eef3-711b-44ad-83b9-ebfcbba90256 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:47:55.247766 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:47:55.261256 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-d738eef3-711b-44ad-83b9-ebfcbba90256 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bd9830058b08458facb66090c8fafcef'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:47:55.333285 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-d738eef3-711b-44ad-83b9-ebfcbba90256 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bd9830058b08458facb66090c8fafcef'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:47:55.333963 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-d738eef3-711b-44ad-83b9-ebfcbba90256 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bd9830058b08458facb66090c8fafcef'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:47:55.346974 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-d738eef3-711b-44ad-83b9-ebfcbba90256 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bd9830058b08458facb66090c8fafcef'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:47:55.354488 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-d738eef3-711b-44ad-83b9-ebfcbba90256 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bd9830058b08458facb66090c8fafcef'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:47:55.360559 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1315/2615] 10.222.0.30 () {58 vars in 1224 bytes} [Tue Sep 6 04:47:54 2022] DELETE /identity/v3/users/bd9830058b08458facb66090c8fafcef => generated 0 bytes in 381 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 04:47:55.365079 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcec79e-dde8-4336-ab4d-233c1cdbfe33 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:47:55.365485 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcec79e-dde8-4336-ab4d-233c1cdbfe33 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:47:55.365826 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcec79e-dde8-4336-ab4d-233c1cdbfe33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:47:55.391155 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-dbcec79e-dde8-4336-ab4d-233c1cdbfe33 None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:47:55.391155 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-dbcec79e-dde8-4336-ab4d-233c1cdbfe33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:47:55.449604 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1301/2616] 10.222.0.30 () {58 vars in 958 bytes} [Tue Sep 6 04:47:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 04:47:55.459301 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d738eef3-711b-44ad-83b9-ebfcbba90256 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:47:55.460705 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-d738eef3-711b-44ad-83b9-ebfcbba90256 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:47:55.520828 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d738eef3-711b-44ad-83b9-ebfcbba90256 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:47:55.524296 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1d045f-89a3-484b-b308-82dd8fe96b9f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90492440f6a6471e8392361c098428aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b707733ce65c42edbc1fd95ad0a1c06c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:47:55.525498 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1d045f-89a3-484b-b308-82dd8fe96b9f service neutron] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:47:55.525770 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1d045f-89a3-484b-b308-82dd8fe96b9f service neutron] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:47:55.526036 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1d045f-89a3-484b-b308-82dd8fe96b9f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:47:55.535176 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c1d045f-89a3-484b-b308-82dd8fe96b9f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:47:55.536594 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c1d045f-89a3-484b-b308-82dd8fe96b9f service neutron] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:47:55.546911 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1316/2617] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Sep 6 04:47:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 04:47:55.907761 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbcec79e-dde8-4336-ab4d-233c1cdbfe33 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:47:55.911848 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-dbcec79e-dde8-4336-ab4d-233c1cdbfe33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:47:55.921683 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbcec79e-dde8-4336-ab4d-233c1cdbfe33 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:47:55.926197 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07eb98c9-9254-4910-b034-0cdcaf0db6c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:47:55.926836 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07eb98c9-9254-4910-b034-0cdcaf0db6c9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:47:55.927108 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07eb98c9-9254-4910-b034-0cdcaf0db6c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:47:55.927374 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07eb98c9-9254-4910-b034-0cdcaf0db6c9 admin admin] PATH_INFO: `/v3/projects/81e530a19a7c459c95377ed0ec6b8565` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:47:55.928481 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07eb98c9-9254-4910-b034-0cdcaf0db6c9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=81e530a19a7c459c95377ed0ec6b8565)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:47:55.930409 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07eb98c9-9254-4910-b034-0cdcaf0db6c9 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:47:56.019676 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-07eb98c9-9254-4910-b034-0cdcaf0db6c9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '81e530a19a7c459c95377ed0ec6b8565', 'request_id': 'req-07eb98c9-9254-4910-b034-0cdcaf0db6c9'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:47:56.088488 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-07eb98c9-9254-4910-b034-0cdcaf0db6c9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '81e530a19a7c459c95377ed0ec6b8565', 'request_id': 'req-07eb98c9-9254-4910-b034-0cdcaf0db6c9'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:47:56.094821 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-07eb98c9-9254-4910-b034-0cdcaf0db6c9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '81e530a19a7c459c95377ed0ec6b8565', 'request_id': 'req-07eb98c9-9254-4910-b034-0cdcaf0db6c9'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:47:56.105023 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-07eb98c9-9254-4910-b034-0cdcaf0db6c9 admin admin] The token cache is being invalidate because project 81e530a19a7c459c95377ed0ec6b8565 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81697) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 04:47:56.105323 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-07eb98c9-9254-4910-b034-0cdcaf0db6c9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:47:56.105952 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-07eb98c9-9254-4910-b034-0cdcaf0db6c9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:47:56.107434 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1302/2618] 10.222.0.30 () {58 vars in 1233 bytes} [Tue Sep 6 04:47:55 2022] DELETE /identity/v3/projects/81e530a19a7c459c95377ed0ec6b8565 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 04:47:56.111468 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1d045f-89a3-484b-b308-82dd8fe96b9f service neutron] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:47:56.122152 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1d045f-89a3-484b-b308-82dd8fe96b9f service neutron] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:47:56.181969 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143e7137-6c74-445f-9db0-d00bc0e68cec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:47:56.182584 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143e7137-6c74-445f-9db0-d00bc0e68cec admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:47:56.182858 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143e7137-6c74-445f-9db0-d00bc0e68cec admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:47:56.183128 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143e7137-6c74-445f-9db0-d00bc0e68cec admin admin] PATH_INFO: `/v3/users/570d49fb567245f29047aa3774df08a9` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:47:56.184273 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143e7137-6c74-445f-9db0-d00bc0e68cec admin admin] RBAC: Authorizing `identity:delete_user(user_id=570d49fb567245f29047aa3774df08a9)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:47:56.186247 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143e7137-6c74-445f-9db0-d00bc0e68cec admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:47:56.369004 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-143e7137-6c74-445f-9db0-d00bc0e68cec admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '570d49fb567245f29047aa3774df08a9'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:47:56.372847 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:47:56.425086 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-143e7137-6c74-445f-9db0-d00bc0e68cec admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '570d49fb567245f29047aa3774df08a9'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:47:56.425788 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-143e7137-6c74-445f-9db0-d00bc0e68cec admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '570d49fb567245f29047aa3774df08a9'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:47:56.439396 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-143e7137-6c74-445f-9db0-d00bc0e68cec admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '570d49fb567245f29047aa3774df08a9'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:47:56.446495 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-143e7137-6c74-445f-9db0-d00bc0e68cec admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '570d49fb567245f29047aa3774df08a9'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:47:56.452552 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1317/2619] 10.222.0.30 () {58 vars in 1224 bytes} [Tue Sep 6 04:47:56 2022] DELETE /identity/v3/users/570d49fb567245f29047aa3774df08a9 => generated 0 bytes in 342 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 04:47:56.916981 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07eb98c9-9254-4910-b034-0cdcaf0db6c9 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:47:56.918491 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-07eb98c9-9254-4910-b034-0cdcaf0db6c9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:47:56.932627 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07eb98c9-9254-4910-b034-0cdcaf0db6c9 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:47:56.977719 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:47:56.990446 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbb5c3f-ac7f-4258-b1eb-d536e1726d2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:47:56.991113 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbb5c3f-ac7f-4258-b1eb-d536e1726d2c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:47:56.991393 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbb5c3f-ac7f-4258-b1eb-d536e1726d2c admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:47:56.991666 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbb5c3f-ac7f-4258-b1eb-d536e1726d2c admin admin] PATH_INFO: `/v3/projects/5bf8e36373d64a0c9766b5297228a40c` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:47:56.992777 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dbb5c3f-ac7f-4258-b1eb-d536e1726d2c admin admin] RBAC: Authorizing `identity:delete_project(project_id=5bf8e36373d64a0c9766b5297228a40c)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:47:56.995225 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dbb5c3f-ac7f-4258-b1eb-d536e1726d2c admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:47:57.097810 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-9dbb5c3f-ac7f-4258-b1eb-d536e1726d2c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5bf8e36373d64a0c9766b5297228a40c', 'request_id': 'req-9dbb5c3f-ac7f-4258-b1eb-d536e1726d2c'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:47:57.154762 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-9dbb5c3f-ac7f-4258-b1eb-d536e1726d2c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5bf8e36373d64a0c9766b5297228a40c', 'request_id': 'req-9dbb5c3f-ac7f-4258-b1eb-d536e1726d2c'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:47:57.161794 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-9dbb5c3f-ac7f-4258-b1eb-d536e1726d2c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5bf8e36373d64a0c9766b5297228a40c', 'request_id': 'req-9dbb5c3f-ac7f-4258-b1eb-d536e1726d2c'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:47:57.173283 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-9dbb5c3f-ac7f-4258-b1eb-d536e1726d2c admin admin] The token cache is being invalidate because project 5bf8e36373d64a0c9766b5297228a40c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81697) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 04:47:57.173599 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-9dbb5c3f-ac7f-4258-b1eb-d536e1726d2c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:47:57.174229 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-9dbb5c3f-ac7f-4258-b1eb-d536e1726d2c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:47:57.175704 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1303/2620] 10.222.0.30 () {58 vars in 1233 bytes} [Tue Sep 6 04:47:56 2022] DELETE /identity/v3/projects/5bf8e36373d64a0c9766b5297228a40c => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 04:47:57.180655 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2340fe-7d4c-4e45-b860-1c74c050545d None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:47:57.181113 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2340fe-7d4c-4e45-b860-1c74c050545d None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:47:57.181234 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2340fe-7d4c-4e45-b860-1c74c050545d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:47:57.201958 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-4e2340fe-7d4c-4e45-b860-1c74c050545d None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:47:57.203156 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-4e2340fe-7d4c-4e45-b860-1c74c050545d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:47:57.262366 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1318/2621] 10.222.0.30 () {58 vars in 958 bytes} [Tue Sep 6 04:47:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 04:47:57.270010 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77778ce-33e1-4336-b264-4bfa950349b4 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:47:57.270300 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77778ce-33e1-4336-b264-4bfa950349b4 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:47:57.270586 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77778ce-33e1-4336-b264-4bfa950349b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:47:57.291579 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-f77778ce-33e1-4336-b264-4bfa950349b4 None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:47:57.292789 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-f77778ce-33e1-4336-b264-4bfa950349b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:47:57.300667 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1304/2622] 10.222.0.30 () {58 vars in 958 bytes} [Tue Sep 6 04:47:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 04:47:57.304832 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2340fe-7d4c-4e45-b860-1c74c050545d None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:47:57.314688 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2340fe-7d4c-4e45-b860-1c74c050545d None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:47:57.319242 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35485c5-44a0-426a-9675-b732165d7095 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:47:57.319843 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35485c5-44a0-426a-9675-b732165d7095 admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:47:57.320126 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35485c5-44a0-426a-9675-b732165d7095 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:47:57.320402 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35485c5-44a0-426a-9675-b732165d7095 admin admin] PATH_INFO: `/v3/domains` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:47:57.321054 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35485c5-44a0-426a-9675-b732165d7095 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:47:57.321473 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35485c5-44a0-426a-9675-b732165d7095 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:47:57.323222 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35485c5-44a0-426a-9675-b732165d7095 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:47:57.334698 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1319/2623] 10.222.0.30 () {58 vars in 1153 bytes} [Tue Sep 6 04:47:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:47:57.339403 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77778ce-33e1-4336-b264-4bfa950349b4 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:47:57.349041 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77778ce-33e1-4336-b264-4bfa950349b4 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:47:57.353765 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c82a6cf-f1c2-432a-adb0-e3fdf59672bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:47:57.354358 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c82a6cf-f1c2-432a-adb0-e3fdf59672bf admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:47:57.354636 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c82a6cf-f1c2-432a-adb0-e3fdf59672bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:47:57.354907 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c82a6cf-f1c2-432a-adb0-e3fdf59672bf admin admin] PATH_INFO: `/v3/projects` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:47:57.355531 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c82a6cf-f1c2-432a-adb0-e3fdf59672bf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:47:57.357511 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c82a6cf-f1c2-432a-adb0-e3fdf59672bf admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:47:57.433192 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1305/2624] 10.222.0.30 () {60 vars in 1153 bytes} [Tue Sep 6 04:47:57 2022] POST /identity/v3/projects => generated 399 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 04:47:57.437394 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35485c5-44a0-426a-9675-b732165d7095 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:47:57.448076 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35485c5-44a0-426a-9675-b732165d7095 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:47:57.508167 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009e27e0-3b4b-4464-8a94-c860fc41f0bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:47:57.509793 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009e27e0-3b4b-4464-8a94-c860fc41f0bb admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:47:57.510076 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009e27e0-3b4b-4464-8a94-c860fc41f0bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:47:57.510353 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009e27e0-3b4b-4464-8a94-c860fc41f0bb admin admin] PATH_INFO: `/v3/users` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:47:57.510989 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009e27e0-3b4b-4464-8a94-c860fc41f0bb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:47:57.512918 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009e27e0-3b4b-4464-8a94-c860fc41f0bb admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:47:57.637254 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1320/2625] 10.222.0.30 () {60 vars in 1144 bytes} [Tue Sep 6 04:47:57 2022] POST /identity/v3/users => generated 300 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 04:47:57.641334 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c82a6cf-f1c2-432a-adb0-e3fdf59672bf admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:47:57.652007 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c82a6cf-f1c2-432a-adb0-e3fdf59672bf admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:47:57.656716 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba211da3-f11d-41fb-830f-070d71a0fa8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:47:57.657341 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba211da3-f11d-41fb-830f-070d71a0fa8d admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:47:57.657622 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba211da3-f11d-41fb-830f-070d71a0fa8d admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:47:57.657892 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba211da3-f11d-41fb-830f-070d71a0fa8d admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:47:57.658552 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba211da3-f11d-41fb-830f-070d71a0fa8d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:47:57.660133 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba211da3-f11d-41fb-830f-070d71a0fa8d admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:47:57.670110 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1306/2626] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Sep 6 04:47:57 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 04:47:57.673829 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009e27e0-3b4b-4464-8a94-c860fc41f0bb admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:47:57.683746 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009e27e0-3b4b-4464-8a94-c860fc41f0bb admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:47:57.688328 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0232f9f-fa84-4216-9c32-1812ab3fee6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:47:57.688916 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0232f9f-fa84-4216-9c32-1812ab3fee6c admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:47:57.691664 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0232f9f-fa84-4216-9c32-1812ab3fee6c admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:47:57.691664 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0232f9f-fa84-4216-9c32-1812ab3fee6c admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:47:57.691664 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0232f9f-fa84-4216-9c32-1812ab3fee6c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:47:57.692263 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0232f9f-fa84-4216-9c32-1812ab3fee6c admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:47:57.704107 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1321/2627] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Sep 6 04:47:57 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 04:47:57.708092 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba211da3-f11d-41fb-830f-070d71a0fa8d admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:47:57.718747 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba211da3-f11d-41fb-830f-070d71a0fa8d admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:47:57.723428 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0867b368-1599-4cf1-94c2-dc305e13a1ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:47:57.724199 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0867b368-1599-4cf1-94c2-dc305e13a1ee admin admin] REQUEST_METHOD: `PUT` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:47:57.724481 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0867b368-1599-4cf1-94c2-dc305e13a1ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:47:57.724752 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0867b368-1599-4cf1-94c2-dc305e13a1ee admin admin] PATH_INFO: `/v3/projects/e9c97926659a44faacd0e1a05bbe09fe/users/87a127b6959d45119ff26caa4417c244/roles/29607b2de39f40d68bb6ceef64b8a732` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:47:57.739992 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0867b368-1599-4cf1-94c2-dc305e13a1ee admin admin] RBAC: Authorizing `identity:create_grant(project_id=e9c97926659a44faacd0e1a05bbe09fe, user_id=87a127b6959d45119ff26caa4417c244, role_id=29607b2de39f40d68bb6ceef64b8a732)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:47:57.742875 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0867b368-1599-4cf1-94c2-dc305e13a1ee admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:47:57.834599 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1307/2628] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Sep 6 04:47:57 2022] PUT /identity/v3/projects/e9c97926659a44faacd0e1a05bbe09fe/users/87a127b6959d45119ff26caa4417c244/roles/29607b2de39f40d68bb6ceef64b8a732 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 04:47:57.846538 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1596764-7b10-4d30-88db-5aad0d257e4c None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:47:57.846824 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1596764-7b10-4d30-88db-5aad0d257e4c None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:47:57.847092 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1596764-7b10-4d30-88db-5aad0d257e4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:47:58.000011 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-f1596764-7b10-4d30-88db-5aad0d257e4c None None] MFA Rules not processed for user `87a127b6959d45119ff26caa4417c244`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:47:58.001137 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-f1596764-7b10-4d30-88db-5aad0d257e4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:47:58.025285 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:47:58.065785 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1322/2629] 10.222.0.30 () {58 vars in 958 bytes} [Tue Sep 6 04:47:57 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 04:47:58.070263 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0867b368-1599-4cf1-94c2-dc305e13a1ee admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:47:58.131483 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0867b368-1599-4cf1-94c2-dc305e13a1ee admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:47:58.136706 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384e4a38-c1b2-4a79-9012-3da3e0a7a77c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:47:58.137340 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384e4a38-c1b2-4a79-9012-3da3e0a7a77c admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:47:58.137620 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384e4a38-c1b2-4a79-9012-3da3e0a7a77c admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:47:58.137890 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384e4a38-c1b2-4a79-9012-3da3e0a7a77c admin admin] PATH_INFO: `/v3/projects` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:47:58.138531 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-384e4a38-c1b2-4a79-9012-3da3e0a7a77c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:47:58.140419 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-384e4a38-c1b2-4a79-9012-3da3e0a7a77c admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:47:58.231742 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:47:58.238551 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1308/2630] 10.222.0.30 () {60 vars in 1153 bytes} [Tue Sep 6 04:47:58 2022] POST /identity/v3/projects => generated 401 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 04:47:58.243827 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1596764-7b10-4d30-88db-5aad0d257e4c None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:47:58.254008 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1596764-7b10-4d30-88db-5aad0d257e4c None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:47:58.309092 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e3567d-a1bb-4b6e-af1d-f088ffd5351c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:47:58.309731 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e3567d-a1bb-4b6e-af1d-f088ffd5351c admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:47:58.310007 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e3567d-a1bb-4b6e-af1d-f088ffd5351c admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:47:58.310275 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e3567d-a1bb-4b6e-af1d-f088ffd5351c admin admin] PATH_INFO: `/v3/users` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:47:58.310929 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e3567d-a1bb-4b6e-af1d-f088ffd5351c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:47:58.312807 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e3567d-a1bb-4b6e-af1d-f088ffd5351c admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:47:58.396917 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1323/2631] 10.222.0.30 () {60 vars in 1144 bytes} [Tue Sep 6 04:47:58 2022] POST /identity/v3/users => generated 307 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 04:47:58.401098 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384e4a38-c1b2-4a79-9012-3da3e0a7a77c admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:47:58.411738 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384e4a38-c1b2-4a79-9012-3da3e0a7a77c admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:47:58.416871 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7874561e-face-4193-846e-7a98216d6144 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:47:58.417499 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7874561e-face-4193-846e-7a98216d6144 admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:47:58.417773 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7874561e-face-4193-846e-7a98216d6144 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:47:58.418040 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7874561e-face-4193-846e-7a98216d6144 admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:47:58.418671 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7874561e-face-4193-846e-7a98216d6144 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:47:58.420234 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7874561e-face-4193-846e-7a98216d6144 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:47:58.430513 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1309/2632] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Sep 6 04:47:58 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 04:47:58.435621 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e3567d-a1bb-4b6e-af1d-f088ffd5351c admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:47:58.445414 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e3567d-a1bb-4b6e-af1d-f088ffd5351c admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:47:58.450664 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a802a06-1bd6-469b-b7f6-b8d6c3eb6442 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:47:58.451286 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a802a06-1bd6-469b-b7f6-b8d6c3eb6442 admin admin] REQUEST_METHOD: `PUT` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:47:58.451565 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a802a06-1bd6-469b-b7f6-b8d6c3eb6442 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:47:58.451838 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a802a06-1bd6-469b-b7f6-b8d6c3eb6442 admin admin] PATH_INFO: `/v3/projects/a8480b64e1c7449c979db5619619735c/users/e448f27f46f34cfc8057e64a706682d6/roles/ea4c814713b149108e488de855b025e4` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:47:58.466822 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a802a06-1bd6-469b-b7f6-b8d6c3eb6442 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a8480b64e1c7449c979db5619619735c, user_id=e448f27f46f34cfc8057e64a706682d6, role_id=ea4c814713b149108e488de855b025e4)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:47:58.469552 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a802a06-1bd6-469b-b7f6-b8d6c3eb6442 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:47:58.566319 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1324/2633] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Sep 6 04:47:58 2022] PUT /identity/v3/projects/a8480b64e1c7449c979db5619619735c/users/e448f27f46f34cfc8057e64a706682d6/roles/ea4c814713b149108e488de855b025e4 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 04:47:58.570268 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7874561e-face-4193-846e-7a98216d6144 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:47:58.580309 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7874561e-face-4193-846e-7a98216d6144 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:47:58.635007 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd4b19f-1473-4c1e-9f61-90c9852167d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:47:58.635611 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd4b19f-1473-4c1e-9f61-90c9852167d7 admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:47:58.635892 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd4b19f-1473-4c1e-9f61-90c9852167d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:47:58.636169 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd4b19f-1473-4c1e-9f61-90c9852167d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:47:58.636802 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd4b19f-1473-4c1e-9f61-90c9852167d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:47:58.638428 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd4b19f-1473-4c1e-9f61-90c9852167d7 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:47:58.647850 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1310/2634] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Sep 6 04:47:58 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 04:47:58.652195 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a802a06-1bd6-469b-b7f6-b8d6c3eb6442 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:47:58.662822 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a802a06-1bd6-469b-b7f6-b8d6c3eb6442 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:47:58.668050 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc56741-3209-4441-9b0d-224aebbdfa29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:47:58.668698 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc56741-3209-4441-9b0d-224aebbdfa29 admin admin] REQUEST_METHOD: `PUT` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:47:58.668977 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc56741-3209-4441-9b0d-224aebbdfa29 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:47:58.669251 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc56741-3209-4441-9b0d-224aebbdfa29 admin admin] PATH_INFO: `/v3/projects/a8480b64e1c7449c979db5619619735c/users/e448f27f46f34cfc8057e64a706682d6/roles/ea4c814713b149108e488de855b025e4` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:47:58.671214 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc56741-3209-4441-9b0d-224aebbdfa29 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a8480b64e1c7449c979db5619619735c, user_id=e448f27f46f34cfc8057e64a706682d6, role_id=ea4c814713b149108e488de855b025e4)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:47:58.673859 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc56741-3209-4441-9b0d-224aebbdfa29 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:47:58.684640 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1325/2635] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Sep 6 04:47:58 2022] PUT /identity/v3/projects/a8480b64e1c7449c979db5619619735c/users/e448f27f46f34cfc8057e64a706682d6/roles/ea4c814713b149108e488de855b025e4 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 04:47:58.694681 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c796e535-52e5-4881-94e1-80ab85692058 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:47:58.694969 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c796e535-52e5-4881-94e1-80ab85692058 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:47:58.695359 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c796e535-52e5-4881-94e1-80ab85692058 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:47:58.722879 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-c796e535-52e5-4881-94e1-80ab85692058 None None] MFA Rules not processed for user `e448f27f46f34cfc8057e64a706682d6`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:47:58.724130 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-c796e535-52e5-4881-94e1-80ab85692058 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:47:58.792312 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1311/2636] 10.222.0.30 () {58 vars in 958 bytes} [Tue Sep 6 04:47:58 2022] POST /identity/v3/auth/tokens => generated 3345 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 04:47:58.803227 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc56741-3209-4441-9b0d-224aebbdfa29 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:47:58.804885 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-8bc56741-3209-4441-9b0d-224aebbdfa29 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:47:58.861751 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc56741-3209-4441-9b0d-224aebbdfa29 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:47:58.865293 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e016777c-f7a8-4d16-a18f-2bc7b47accb8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd09b5ab675634a2eb4b03755743f7d0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b707733ce65c42edbc1fd95ad0a1c06c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:47:58.865919 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e016777c-f7a8-4d16-a18f-2bc7b47accb8 service cinder] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:47:58.866198 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e016777c-f7a8-4d16-a18f-2bc7b47accb8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:47:58.868746 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e016777c-f7a8-4d16-a18f-2bc7b47accb8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:47:58.876218 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e016777c-f7a8-4d16-a18f-2bc7b47accb8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:47:58.877674 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e016777c-f7a8-4d16-a18f-2bc7b47accb8 service cinder] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:47:58.887961 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1326/2637] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Sep 6 04:47:58 2022] GET /identity/v3/auth/tokens => generated 3345 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 04:47:59.110100 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c796e535-52e5-4881-94e1-80ab85692058 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:47:59.123752 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c796e535-52e5-4881-94e1-80ab85692058 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:47:59.123955 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785c0151-b556-43af-9c8e-e25d67f5476b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd09b5ab675634a2eb4b03755743f7d0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b707733ce65c42edbc1fd95ad0a1c06c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:47:59.124524 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785c0151-b556-43af-9c8e-e25d67f5476b service cinder] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:47:59.124802 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785c0151-b556-43af-9c8e-e25d67f5476b service cinder] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:47:59.125075 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785c0151-b556-43af-9c8e-e25d67f5476b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:47:59.186531 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785c0151-b556-43af-9c8e-e25d67f5476b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:47:59.187955 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785c0151-b556-43af-9c8e-e25d67f5476b service cinder] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:47:59.196720 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1312/2638] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Sep 6 04:47:59 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 04:48:04.178238 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e016777c-f7a8-4d16-a18f-2bc7b47accb8 service cinder] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:48:04.199055 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:48:04.238735 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e016777c-f7a8-4d16-a18f-2bc7b47accb8 service cinder] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:48:04.243397 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae76676-9694-4706-8644-08fa40f5d30f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:48:04.247858 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae76676-9694-4706-8644-08fa40f5d30f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:48:04.247858 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae76676-9694-4706-8644-08fa40f5d30f admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:48:04.247858 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae76676-9694-4706-8644-08fa40f5d30f admin admin] PATH_INFO: `/v3/users/87a127b6959d45119ff26caa4417c244` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:48:04.247858 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae76676-9694-4706-8644-08fa40f5d30f admin admin] RBAC: Authorizing `identity:delete_user(user_id=87a127b6959d45119ff26caa4417c244)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:48:04.248781 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae76676-9694-4706-8644-08fa40f5d30f admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:48:04.389253 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-aae76676-9694-4706-8644-08fa40f5d30f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '87a127b6959d45119ff26caa4417c244'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:48:04.442097 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-aae76676-9694-4706-8644-08fa40f5d30f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '87a127b6959d45119ff26caa4417c244'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:48:04.442712 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-aae76676-9694-4706-8644-08fa40f5d30f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '87a127b6959d45119ff26caa4417c244'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:48:04.455182 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-aae76676-9694-4706-8644-08fa40f5d30f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '87a127b6959d45119ff26caa4417c244'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:48:04.462004 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-aae76676-9694-4706-8644-08fa40f5d30f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '87a127b6959d45119ff26caa4417c244'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:48:04.467911 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1327/2639] 10.222.0.30 () {58 vars in 1224 bytes} [Tue Sep 6 04:48:04 2022] DELETE /identity/v3/users/87a127b6959d45119ff26caa4417c244 => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 04:48:04.472411 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fabc176-94b1-4148-97a5-fbb6458f786a None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:48:04.472933 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fabc176-94b1-4148-97a5-fbb6458f786a None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:48:04.472933 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fabc176-94b1-4148-97a5-fbb6458f786a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:48:04.486531 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:48:04.495216 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-6fabc176-94b1-4148-97a5-fbb6458f786a None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:48:04.495216 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-6fabc176-94b1-4148-97a5-fbb6458f786a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:48:04.553325 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1313/2640] 10.222.0.30 () {58 vars in 958 bytes} [Tue Sep 6 04:48:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 04:48:04.563646 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae76676-9694-4706-8644-08fa40f5d30f admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:48:04.565034 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-aae76676-9694-4706-8644-08fa40f5d30f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:48:04.619979 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae76676-9694-4706-8644-08fa40f5d30f admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:48:04.624153 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54644c1a-2b2b-491e-8962-f01e107f30b5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90492440f6a6471e8392361c098428aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b707733ce65c42edbc1fd95ad0a1c06c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:48:04.624746 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54644c1a-2b2b-491e-8962-f01e107f30b5 service neutron] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:48:04.625019 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54644c1a-2b2b-491e-8962-f01e107f30b5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:48:04.625284 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54644c1a-2b2b-491e-8962-f01e107f30b5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:48:04.634091 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54644c1a-2b2b-491e-8962-f01e107f30b5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:48:04.635502 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54644c1a-2b2b-491e-8962-f01e107f30b5 service neutron] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:48:04.645008 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1328/2641] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Sep 6 04:48:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 04:48:05.043349 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fabc176-94b1-4148-97a5-fbb6458f786a None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:48:05.045425 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-6fabc176-94b1-4148-97a5-fbb6458f786a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:48:05.057995 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fabc176-94b1-4148-97a5-fbb6458f786a None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:48:05.064368 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1619f5-7b51-49c0-b779-05e51d2d8c9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:48:05.065014 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1619f5-7b51-49c0-b779-05e51d2d8c9c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:48:05.065292 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1619f5-7b51-49c0-b779-05e51d2d8c9c admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:48:05.065600 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1619f5-7b51-49c0-b779-05e51d2d8c9c admin admin] PATH_INFO: `/v3/projects/e9c97926659a44faacd0e1a05bbe09fe` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:48:05.066750 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d1619f5-7b51-49c0-b779-05e51d2d8c9c admin admin] RBAC: Authorizing `identity:delete_project(project_id=e9c97926659a44faacd0e1a05bbe09fe)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:48:05.068713 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d1619f5-7b51-49c0-b779-05e51d2d8c9c admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:48:05.150502 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-1d1619f5-7b51-49c0-b779-05e51d2d8c9c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e9c97926659a44faacd0e1a05bbe09fe', 'request_id': 'req-1d1619f5-7b51-49c0-b779-05e51d2d8c9c'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:48:05.201892 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-1d1619f5-7b51-49c0-b779-05e51d2d8c9c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e9c97926659a44faacd0e1a05bbe09fe', 'request_id': 'req-1d1619f5-7b51-49c0-b779-05e51d2d8c9c'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:48:05.208273 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-1d1619f5-7b51-49c0-b779-05e51d2d8c9c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e9c97926659a44faacd0e1a05bbe09fe', 'request_id': 'req-1d1619f5-7b51-49c0-b779-05e51d2d8c9c'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:48:05.219062 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-1d1619f5-7b51-49c0-b779-05e51d2d8c9c admin admin] The token cache is being invalidate because project e9c97926659a44faacd0e1a05bbe09fe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81697) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 04:48:05.219353 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-1d1619f5-7b51-49c0-b779-05e51d2d8c9c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:48:05.219983 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-1d1619f5-7b51-49c0-b779-05e51d2d8c9c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:48:05.221600 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1314/2642] 10.222.0.30 () {58 vars in 1233 bytes} [Tue Sep 6 04:48:05 2022] DELETE /identity/v3/projects/e9c97926659a44faacd0e1a05bbe09fe => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 04:48:05.225585 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54644c1a-2b2b-491e-8962-f01e107f30b5 service neutron] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:48:05.235692 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54644c1a-2b2b-491e-8962-f01e107f30b5 service neutron] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:48:05.274820 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:48:05.291737 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6833577-bdfd-40d2-996e-54c8803fa6e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:48:05.292407 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6833577-bdfd-40d2-996e-54c8803fa6e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:48:05.292698 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6833577-bdfd-40d2-996e-54c8803fa6e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:48:05.292971 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6833577-bdfd-40d2-996e-54c8803fa6e4 admin admin] PATH_INFO: `/v3/users/e448f27f46f34cfc8057e64a706682d6` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:48:05.294674 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6833577-bdfd-40d2-996e-54c8803fa6e4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e448f27f46f34cfc8057e64a706682d6)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:48:05.296625 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6833577-bdfd-40d2-996e-54c8803fa6e4 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:48:05.484323 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-d6833577-bdfd-40d2-996e-54c8803fa6e4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e448f27f46f34cfc8057e64a706682d6'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:48:05.550832 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-d6833577-bdfd-40d2-996e-54c8803fa6e4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e448f27f46f34cfc8057e64a706682d6'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:48:05.551468 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-d6833577-bdfd-40d2-996e-54c8803fa6e4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e448f27f46f34cfc8057e64a706682d6'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:48:05.564559 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-d6833577-bdfd-40d2-996e-54c8803fa6e4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e448f27f46f34cfc8057e64a706682d6'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:48:05.571534 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-d6833577-bdfd-40d2-996e-54c8803fa6e4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e448f27f46f34cfc8057e64a706682d6'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:48:05.578182 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1329/2643] 10.222.0.30 () {58 vars in 1224 bytes} [Tue Sep 6 04:48:05 2022] DELETE /identity/v3/users/e448f27f46f34cfc8057e64a706682d6 => generated 0 bytes in 353 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 04:48:05.941046 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1619f5-7b51-49c0-b779-05e51d2d8c9c admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:48:05.941511 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-1d1619f5-7b51-49c0-b779-05e51d2d8c9c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:48:05.954245 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1619f5-7b51-49c0-b779-05e51d2d8c9c admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:48:06.010400 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ec49fa-773a-4e9e-9dbb-f0cc4051d9b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:48:06.011025 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ec49fa-773a-4e9e-9dbb-f0cc4051d9b1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:48:06.011299 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ec49fa-773a-4e9e-9dbb-f0cc4051d9b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:48:06.011569 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ec49fa-773a-4e9e-9dbb-f0cc4051d9b1 admin admin] PATH_INFO: `/v3/projects/a8480b64e1c7449c979db5619619735c` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:48:06.012670 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ec49fa-773a-4e9e-9dbb-f0cc4051d9b1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a8480b64e1c7449c979db5619619735c)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:48:06.014672 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ec49fa-773a-4e9e-9dbb-f0cc4051d9b1 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:48:06.149170 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:48:06.164675 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-14ec49fa-773a-4e9e-9dbb-f0cc4051d9b1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a8480b64e1c7449c979db5619619735c', 'request_id': 'req-14ec49fa-773a-4e9e-9dbb-f0cc4051d9b1'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:48:06.230444 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-14ec49fa-773a-4e9e-9dbb-f0cc4051d9b1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a8480b64e1c7449c979db5619619735c', 'request_id': 'req-14ec49fa-773a-4e9e-9dbb-f0cc4051d9b1'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:48:06.237702 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-14ec49fa-773a-4e9e-9dbb-f0cc4051d9b1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a8480b64e1c7449c979db5619619735c', 'request_id': 'req-14ec49fa-773a-4e9e-9dbb-f0cc4051d9b1'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:48:06.248509 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-14ec49fa-773a-4e9e-9dbb-f0cc4051d9b1 admin admin] The token cache is being invalidate because project a8480b64e1c7449c979db5619619735c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81697) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 04:48:06.248799 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-14ec49fa-773a-4e9e-9dbb-f0cc4051d9b1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:48:06.249463 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-14ec49fa-773a-4e9e-9dbb-f0cc4051d9b1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:48:06.250934 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1315/2644] 10.222.0.30 () {58 vars in 1233 bytes} [Tue Sep 6 04:48:05 2022] DELETE /identity/v3/projects/a8480b64e1c7449c979db5619619735c => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 04:48:06.255871 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d8367c-a296-4ca5-8b9f-d694297fe9df None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:48:06.256353 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d8367c-a296-4ca5-8b9f-d694297fe9df None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:48:06.256490 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d8367c-a296-4ca5-8b9f-d694297fe9df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:48:06.276864 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-b2d8367c-a296-4ca5-8b9f-d694297fe9df None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:48:06.278113 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-b2d8367c-a296-4ca5-8b9f-d694297fe9df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:48:06.339425 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1330/2645] 10.222.0.30 () {58 vars in 958 bytes} [Tue Sep 6 04:48:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 04:48:06.348030 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c74c25-0730-45c0-bdb7-09e06a0522cf None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:48:06.348320 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c74c25-0730-45c0-bdb7-09e06a0522cf None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:48:06.348599 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c74c25-0730-45c0-bdb7-09e06a0522cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:48:06.370087 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-60c74c25-0730-45c0-bdb7-09e06a0522cf None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:48:06.371342 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-60c74c25-0730-45c0-bdb7-09e06a0522cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:48:06.383361 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1316/2646] 10.222.0.30 () {58 vars in 958 bytes} [Tue Sep 6 04:48:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 04:48:06.387856 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d8367c-a296-4ca5-8b9f-d694297fe9df None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:48:06.397480 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d8367c-a296-4ca5-8b9f-d694297fe9df None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:48:06.401874 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1cc9be1-6e48-4e80-a197-7741485931f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:48:06.402461 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cc9be1-6e48-4e80-a197-7741485931f6 admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:48:06.402735 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cc9be1-6e48-4e80-a197-7741485931f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:48:06.403005 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cc9be1-6e48-4e80-a197-7741485931f6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:48:06.404142 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1cc9be1-6e48-4e80-a197-7741485931f6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:48:06.404438 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1cc9be1-6e48-4e80-a197-7741485931f6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:48:06.406306 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1cc9be1-6e48-4e80-a197-7741485931f6 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:48:06.417575 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1331/2647] 10.222.0.30 () {58 vars in 1153 bytes} [Tue Sep 6 04:48:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:48:06.423567 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c74c25-0730-45c0-bdb7-09e06a0522cf None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:48:06.433126 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c74c25-0730-45c0-bdb7-09e06a0522cf None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:48:06.438159 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2877a58f-61df-4054-94d0-4bc0dbdc4a1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:48:06.438762 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2877a58f-61df-4054-94d0-4bc0dbdc4a1a admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:48:06.439040 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2877a58f-61df-4054-94d0-4bc0dbdc4a1a admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:48:06.439310 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2877a58f-61df-4054-94d0-4bc0dbdc4a1a admin admin] PATH_INFO: `/v3/projects` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:48:06.439941 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2877a58f-61df-4054-94d0-4bc0dbdc4a1a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:48:06.441867 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2877a58f-61df-4054-94d0-4bc0dbdc4a1a admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:48:06.540774 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1317/2648] 10.222.0.30 () {60 vars in 1153 bytes} [Tue Sep 6 04:48:06 2022] POST /identity/v3/projects => generated 383 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 04:48:06.545335 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1cc9be1-6e48-4e80-a197-7741485931f6 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:48:06.555314 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1cc9be1-6e48-4e80-a197-7741485931f6 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:48:06.599468 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:48:06.610978 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd3892c-cc55-447d-b688-696955e3d942 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:48:06.611579 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd3892c-cc55-447d-b688-696955e3d942 admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:48:06.611855 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd3892c-cc55-447d-b688-696955e3d942 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:48:06.612241 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd3892c-cc55-447d-b688-696955e3d942 admin admin] PATH_INFO: `/v3/users` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:48:06.612733 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd3892c-cc55-447d-b688-696955e3d942 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:48:06.614671 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd3892c-cc55-447d-b688-696955e3d942 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:48:06.675987 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1332/2649] 10.222.0.30 () {60 vars in 1144 bytes} [Tue Sep 6 04:48:06 2022] POST /identity/v3/users => generated 292 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 04:48:06.680291 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2877a58f-61df-4054-94d0-4bc0dbdc4a1a admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:48:06.691019 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2877a58f-61df-4054-94d0-4bc0dbdc4a1a admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:48:06.695832 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c3cc1f-6133-4166-a186-b86bacbccd38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:48:06.696421 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c3cc1f-6133-4166-a186-b86bacbccd38 admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:48:06.696696 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c3cc1f-6133-4166-a186-b86bacbccd38 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:48:06.696963 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c3cc1f-6133-4166-a186-b86bacbccd38 admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:48:06.697730 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c3cc1f-6133-4166-a186-b86bacbccd38 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:48:06.699363 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c3cc1f-6133-4166-a186-b86bacbccd38 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:48:06.710500 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1318/2650] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Sep 6 04:48:06 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 04:48:06.714575 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd3892c-cc55-447d-b688-696955e3d942 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:48:06.725925 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd3892c-cc55-447d-b688-696955e3d942 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:48:06.730586 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa638d8-70f6-44b5-9349-596bc1b0ec15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:48:06.731181 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa638d8-70f6-44b5-9349-596bc1b0ec15 admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:48:06.731459 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa638d8-70f6-44b5-9349-596bc1b0ec15 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:48:06.731847 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa638d8-70f6-44b5-9349-596bc1b0ec15 admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:48:06.732685 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa638d8-70f6-44b5-9349-596bc1b0ec15 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:48:06.734300 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa638d8-70f6-44b5-9349-596bc1b0ec15 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:48:06.744647 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1333/2651] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Sep 6 04:48:06 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 04:48:06.748946 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c3cc1f-6133-4166-a186-b86bacbccd38 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:48:06.759028 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c3cc1f-6133-4166-a186-b86bacbccd38 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:48:06.764299 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d92a49-1371-4494-b990-81f6cfb2aef2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:48:06.764936 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d92a49-1371-4494-b990-81f6cfb2aef2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:48:06.765208 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d92a49-1371-4494-b990-81f6cfb2aef2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:48:06.765548 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d92a49-1371-4494-b990-81f6cfb2aef2 admin admin] PATH_INFO: `/v3/projects/299a9dd81a0b4215a7392ac1c5b46ce2/users/244a4e42bf6b4453a357192ad6207b46/roles/29607b2de39f40d68bb6ceef64b8a732` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:48:06.780981 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d92a49-1371-4494-b990-81f6cfb2aef2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=299a9dd81a0b4215a7392ac1c5b46ce2, user_id=244a4e42bf6b4453a357192ad6207b46, role_id=29607b2de39f40d68bb6ceef64b8a732)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:48:06.783721 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d92a49-1371-4494-b990-81f6cfb2aef2 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:48:06.842472 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1319/2652] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Sep 6 04:48:06 2022] PUT /identity/v3/projects/299a9dd81a0b4215a7392ac1c5b46ce2/users/244a4e42bf6b4453a357192ad6207b46/roles/29607b2de39f40d68bb6ceef64b8a732 => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 04:48:06.854045 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce41eb1c-3220-4d0f-b296-db57d2368496 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:48:06.854352 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce41eb1c-3220-4d0f-b296-db57d2368496 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:48:06.854629 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce41eb1c-3220-4d0f-b296-db57d2368496 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:48:06.882266 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-ce41eb1c-3220-4d0f-b296-db57d2368496 None None] MFA Rules not processed for user `244a4e42bf6b4453a357192ad6207b46`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:48:06.883480 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-ce41eb1c-3220-4d0f-b296-db57d2368496 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:48:06.948346 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1334/2653] 10.222.0.30 () {58 vars in 958 bytes} [Tue Sep 6 04:48:06 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 04:48:06.953540 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d92a49-1371-4494-b990-81f6cfb2aef2 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:48:07.017407 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d92a49-1371-4494-b990-81f6cfb2aef2 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:48:07.022388 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18617e8-4432-4823-8082-b64c85cfc6bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:48:07.023000 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18617e8-4432-4823-8082-b64c85cfc6bb admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:48:07.023279 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18617e8-4432-4823-8082-b64c85cfc6bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:48:07.023549 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18617e8-4432-4823-8082-b64c85cfc6bb admin admin] PATH_INFO: `/v3/projects` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:48:07.024189 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18617e8-4432-4823-8082-b64c85cfc6bb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:48:07.026111 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18617e8-4432-4823-8082-b64c85cfc6bb admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:48:07.096450 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1320/2654] 10.222.0.30 () {60 vars in 1153 bytes} [Tue Sep 6 04:48:06 2022] POST /identity/v3/projects => generated 383 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 04:48:07.100124 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce41eb1c-3220-4d0f-b296-db57d2368496 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:48:07.112264 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce41eb1c-3220-4d0f-b296-db57d2368496 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:48:07.170358 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941f203c-4ac9-4334-b740-d973f8dde3ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:48:07.170973 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941f203c-4ac9-4334-b740-d973f8dde3ee admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:48:07.171254 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941f203c-4ac9-4334-b740-d973f8dde3ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:48:07.171524 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941f203c-4ac9-4334-b740-d973f8dde3ee admin admin] PATH_INFO: `/v3/users` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:48:07.172167 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941f203c-4ac9-4334-b740-d973f8dde3ee admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:48:07.174120 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941f203c-4ac9-4334-b740-d973f8dde3ee admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:48:07.252951 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1335/2655] 10.222.0.30 () {60 vars in 1144 bytes} [Tue Sep 6 04:48:07 2022] POST /identity/v3/users => generated 298 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 04:48:07.257046 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18617e8-4432-4823-8082-b64c85cfc6bb admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:48:07.268803 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18617e8-4432-4823-8082-b64c85cfc6bb admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:48:07.273330 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7e792f-86bc-4913-8d0c-616ee2c4ce8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:48:07.273944 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7e792f-86bc-4913-8d0c-616ee2c4ce8d admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:48:07.275489 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7e792f-86bc-4913-8d0c-616ee2c4ce8d admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:48:07.275489 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7e792f-86bc-4913-8d0c-616ee2c4ce8d admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:48:07.275489 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7e792f-86bc-4913-8d0c-616ee2c4ce8d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:48:07.276723 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7e792f-86bc-4913-8d0c-616ee2c4ce8d admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:48:07.287943 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1321/2656] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Sep 6 04:48:07 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 06 04:48:07.292190 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941f203c-4ac9-4334-b740-d973f8dde3ee admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:48:07.303720 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941f203c-4ac9-4334-b740-d973f8dde3ee admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:48:07.311203 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efac7e3b-3be3-4221-8632-26568d678b74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:48:07.312056 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efac7e3b-3be3-4221-8632-26568d678b74 admin admin] REQUEST_METHOD: `PUT` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:48:07.312325 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efac7e3b-3be3-4221-8632-26568d678b74 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:48:07.312600 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efac7e3b-3be3-4221-8632-26568d678b74 admin admin] PATH_INFO: `/v3/projects/8029044c500742ab907004dc94b3cb6e/users/bedc963db7bf46f3be6d0cbc82178c6a/roles/ea4c814713b149108e488de855b025e4` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:48:07.327888 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efac7e3b-3be3-4221-8632-26568d678b74 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8029044c500742ab907004dc94b3cb6e, user_id=bedc963db7bf46f3be6d0cbc82178c6a, role_id=ea4c814713b149108e488de855b025e4)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:48:07.330548 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efac7e3b-3be3-4221-8632-26568d678b74 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:48:07.400181 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1336/2657] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Sep 6 04:48:07 2022] PUT /identity/v3/projects/8029044c500742ab907004dc94b3cb6e/users/bedc963db7bf46f3be6d0cbc82178c6a/roles/ea4c814713b149108e488de855b025e4 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 04:48:07.404490 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7e792f-86bc-4913-8d0c-616ee2c4ce8d admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:48:07.414711 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7e792f-86bc-4913-8d0c-616ee2c4ce8d admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:48:07.446652 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:48:07.475224 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd50a2e-5c5e-4bb4-bb8c-a246be9862e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:48:07.475858 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd50a2e-5c5e-4bb4-bb8c-a246be9862e9 admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:48:07.476137 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd50a2e-5c5e-4bb4-bb8c-a246be9862e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:48:07.476407 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd50a2e-5c5e-4bb4-bb8c-a246be9862e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:48:07.477057 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd50a2e-5c5e-4bb4-bb8c-a246be9862e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:48:07.478684 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd50a2e-5c5e-4bb4-bb8c-a246be9862e9 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:48:07.489639 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1322/2658] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Sep 6 04:48:07 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 06 04:48:07.493711 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efac7e3b-3be3-4221-8632-26568d678b74 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:48:07.504699 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efac7e3b-3be3-4221-8632-26568d678b74 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:48:07.509521 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a871f7-523e-4ea4-ad92-34537287ae83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:48:07.510181 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a871f7-523e-4ea4-ad92-34537287ae83 admin admin] REQUEST_METHOD: `PUT` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:48:07.510454 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a871f7-523e-4ea4-ad92-34537287ae83 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:48:07.510749 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a871f7-523e-4ea4-ad92-34537287ae83 admin admin] PATH_INFO: `/v3/projects/8029044c500742ab907004dc94b3cb6e/users/bedc963db7bf46f3be6d0cbc82178c6a/roles/ea4c814713b149108e488de855b025e4` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:48:07.512667 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a871f7-523e-4ea4-ad92-34537287ae83 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8029044c500742ab907004dc94b3cb6e, user_id=bedc963db7bf46f3be6d0cbc82178c6a, role_id=ea4c814713b149108e488de855b025e4)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:48:07.515713 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a871f7-523e-4ea4-ad92-34537287ae83 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:48:07.526440 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1337/2659] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Sep 6 04:48:07 2022] PUT /identity/v3/projects/8029044c500742ab907004dc94b3cb6e/users/bedc963db7bf46f3be6d0cbc82178c6a/roles/ea4c814713b149108e488de855b025e4 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 04:48:07.537000 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae865f9-9e44-421a-bf58-9bc25713605a None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:48:07.537391 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae865f9-9e44-421a-bf58-9bc25713605a None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:48:07.537766 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae865f9-9e44-421a-bf58-9bc25713605a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:48:07.567958 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-fae865f9-9e44-421a-bf58-9bc25713605a None None] MFA Rules not processed for user `bedc963db7bf46f3be6d0cbc82178c6a`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:48:07.569218 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-fae865f9-9e44-421a-bf58-9bc25713605a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:48:07.634463 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1323/2660] 10.222.0.30 () {58 vars in 958 bytes} [Tue Sep 6 04:48:07 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 04:48:07.645390 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a871f7-523e-4ea4-ad92-34537287ae83 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:48:07.647082 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-22a871f7-523e-4ea4-ad92-34537287ae83 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:48:07.658580 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:48:07.703908 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a871f7-523e-4ea4-ad92-34537287ae83 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:48:07.707507 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7758cd93-af6f-4d81-b73b-6dd7bbd1382a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd09b5ab675634a2eb4b03755743f7d0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b707733ce65c42edbc1fd95ad0a1c06c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:48:07.708158 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7758cd93-af6f-4d81-b73b-6dd7bbd1382a service cinder] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:48:07.708433 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7758cd93-af6f-4d81-b73b-6dd7bbd1382a service cinder] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:48:07.708695 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7758cd93-af6f-4d81-b73b-6dd7bbd1382a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:48:07.718408 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7758cd93-af6f-4d81-b73b-6dd7bbd1382a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:48:07.719827 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7758cd93-af6f-4d81-b73b-6dd7bbd1382a service cinder] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:48:07.729216 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1338/2661] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Sep 6 04:48:07 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 04:48:07.835107 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae865f9-9e44-421a-bf58-9bc25713605a None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:48:07.846220 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae865f9-9e44-421a-bf58-9bc25713605a None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:48:07.849959 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f1c16c-409f-4364-ac68-92190c05f5d8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd09b5ab675634a2eb4b03755743f7d0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b707733ce65c42edbc1fd95ad0a1c06c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:48:07.850570 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f1c16c-409f-4364-ac68-92190c05f5d8 service cinder] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:48:07.850869 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f1c16c-409f-4364-ac68-92190c05f5d8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:48:07.851136 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f1c16c-409f-4364-ac68-92190c05f5d8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:48:07.906919 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f1c16c-409f-4364-ac68-92190c05f5d8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:48:07.908265 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f1c16c-409f-4364-ac68-92190c05f5d8 service cinder] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:48:07.916917 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1324/2662] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Sep 6 04:48:07 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 04:48:11.074140 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7758cd93-af6f-4d81-b73b-6dd7bbd1382a service cinder] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:48:11.142775 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7758cd93-af6f-4d81-b73b-6dd7bbd1382a service cinder] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:48:11.148294 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9fcdeb-76f6-43b5-bc51-eb2b6cbd2bdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:48:11.149035 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9fcdeb-76f6-43b5-bc51-eb2b6cbd2bdf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:48:11.149443 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9fcdeb-76f6-43b5-bc51-eb2b6cbd2bdf admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:48:11.149805 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9fcdeb-76f6-43b5-bc51-eb2b6cbd2bdf admin admin] PATH_INFO: `/v3/users/244a4e42bf6b4453a357192ad6207b46` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:48:11.152078 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9fcdeb-76f6-43b5-bc51-eb2b6cbd2bdf admin admin] RBAC: Authorizing `identity:delete_user(user_id=244a4e42bf6b4453a357192ad6207b46)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:48:11.154234 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9fcdeb-76f6-43b5-bc51-eb2b6cbd2bdf admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:48:11.364744 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-6d9fcdeb-76f6-43b5-bc51-eb2b6cbd2bdf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '244a4e42bf6b4453a357192ad6207b46'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:48:11.440622 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-6d9fcdeb-76f6-43b5-bc51-eb2b6cbd2bdf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '244a4e42bf6b4453a357192ad6207b46'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:48:11.441182 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-6d9fcdeb-76f6-43b5-bc51-eb2b6cbd2bdf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '244a4e42bf6b4453a357192ad6207b46'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:48:11.454250 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-6d9fcdeb-76f6-43b5-bc51-eb2b6cbd2bdf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '244a4e42bf6b4453a357192ad6207b46'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:48:11.461020 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-6d9fcdeb-76f6-43b5-bc51-eb2b6cbd2bdf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '244a4e42bf6b4453a357192ad6207b46'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:48:11.467553 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1339/2663] 10.222.0.30 () {58 vars in 1224 bytes} [Tue Sep 6 04:48:11 2022] DELETE /identity/v3/users/244a4e42bf6b4453a357192ad6207b46 => generated 0 bytes in 394 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 04:48:11.472078 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7478051-c6e3-4215-b6be-9f4f6d553b2b None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:48:11.472572 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7478051-c6e3-4215-b6be-9f4f6d553b2b None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:48:11.472676 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7478051-c6e3-4215-b6be-9f4f6d553b2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:48:11.497226 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-d7478051-c6e3-4215-b6be-9f4f6d553b2b None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:48:11.499135 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-d7478051-c6e3-4215-b6be-9f4f6d553b2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:48:11.563963 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:48:11.579867 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1325/2664] 10.222.0.30 () {58 vars in 958 bytes} [Tue Sep 6 04:48:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 04:48:11.589472 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9fcdeb-76f6-43b5-bc51-eb2b6cbd2bdf admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:48:11.591476 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-6d9fcdeb-76f6-43b5-bc51-eb2b6cbd2bdf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:48:11.627056 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:48:11.649634 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9fcdeb-76f6-43b5-bc51-eb2b6cbd2bdf admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:48:11.653902 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad63180-6392-4a91-99a4-126fb14b8528 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90492440f6a6471e8392361c098428aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b707733ce65c42edbc1fd95ad0a1c06c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:48:11.654502 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad63180-6392-4a91-99a4-126fb14b8528 service neutron] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:48:11.654780 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad63180-6392-4a91-99a4-126fb14b8528 service neutron] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:48:11.655054 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad63180-6392-4a91-99a4-126fb14b8528 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:48:11.664908 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad63180-6392-4a91-99a4-126fb14b8528 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:48:11.666408 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad63180-6392-4a91-99a4-126fb14b8528 service neutron] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:48:11.678242 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1340/2665] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Sep 6 04:48:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 04:48:12.066989 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7478051-c6e3-4215-b6be-9f4f6d553b2b None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:48:12.068490 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-d7478051-c6e3-4215-b6be-9f4f6d553b2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:48:12.096958 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7478051-c6e3-4215-b6be-9f4f6d553b2b None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:48:12.101728 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523feca1-f4a6-44be-8fcf-01c2da19a8e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:48:12.102374 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523feca1-f4a6-44be-8fcf-01c2da19a8e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:48:12.102655 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523feca1-f4a6-44be-8fcf-01c2da19a8e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:48:12.103267 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523feca1-f4a6-44be-8fcf-01c2da19a8e4 admin admin] PATH_INFO: `/v3/projects/299a9dd81a0b4215a7392ac1c5b46ce2` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:48:12.104012 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523feca1-f4a6-44be-8fcf-01c2da19a8e4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=299a9dd81a0b4215a7392ac1c5b46ce2)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:48:12.105930 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523feca1-f4a6-44be-8fcf-01c2da19a8e4 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:48:12.206374 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-523feca1-f4a6-44be-8fcf-01c2da19a8e4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '299a9dd81a0b4215a7392ac1c5b46ce2', 'request_id': 'req-523feca1-f4a6-44be-8fcf-01c2da19a8e4'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:48:12.259143 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-523feca1-f4a6-44be-8fcf-01c2da19a8e4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '299a9dd81a0b4215a7392ac1c5b46ce2', 'request_id': 'req-523feca1-f4a6-44be-8fcf-01c2da19a8e4'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:48:12.267514 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-523feca1-f4a6-44be-8fcf-01c2da19a8e4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '299a9dd81a0b4215a7392ac1c5b46ce2', 'request_id': 'req-523feca1-f4a6-44be-8fcf-01c2da19a8e4'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:48:12.279203 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-523feca1-f4a6-44be-8fcf-01c2da19a8e4 admin admin] The token cache is being invalidate because project 299a9dd81a0b4215a7392ac1c5b46ce2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81697) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 04:48:12.279534 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-523feca1-f4a6-44be-8fcf-01c2da19a8e4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:48:12.280197 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-523feca1-f4a6-44be-8fcf-01c2da19a8e4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:48:12.282032 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1326/2666] 10.222.0.30 () {58 vars in 1233 bytes} [Tue Sep 6 04:48:12 2022] DELETE /identity/v3/projects/299a9dd81a0b4215a7392ac1c5b46ce2 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 04:48:12.286181 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad63180-6392-4a91-99a4-126fb14b8528 service neutron] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:48:12.296798 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad63180-6392-4a91-99a4-126fb14b8528 service neutron] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:48:12.354258 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64d2105-a76b-476d-b295-60b4c41d13f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:48:12.354888 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64d2105-a76b-476d-b295-60b4c41d13f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:48:12.355163 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64d2105-a76b-476d-b295-60b4c41d13f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:48:12.355434 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64d2105-a76b-476d-b295-60b4c41d13f5 admin admin] PATH_INFO: `/v3/users/bedc963db7bf46f3be6d0cbc82178c6a` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:48:12.357170 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a64d2105-a76b-476d-b295-60b4c41d13f5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bedc963db7bf46f3be6d0cbc82178c6a)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:48:12.359458 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a64d2105-a76b-476d-b295-60b4c41d13f5 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:48:12.531454 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-a64d2105-a76b-476d-b295-60b4c41d13f5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bedc963db7bf46f3be6d0cbc82178c6a'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:48:12.596487 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-a64d2105-a76b-476d-b295-60b4c41d13f5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bedc963db7bf46f3be6d0cbc82178c6a'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:48:12.597127 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-a64d2105-a76b-476d-b295-60b4c41d13f5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bedc963db7bf46f3be6d0cbc82178c6a'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:48:12.611745 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-a64d2105-a76b-476d-b295-60b4c41d13f5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bedc963db7bf46f3be6d0cbc82178c6a'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:48:12.618811 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-a64d2105-a76b-476d-b295-60b4c41d13f5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bedc963db7bf46f3be6d0cbc82178c6a'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:48:12.625552 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1341/2667] 10.222.0.30 () {58 vars in 1224 bytes} [Tue Sep 6 04:48:12 2022] DELETE /identity/v3/users/bedc963db7bf46f3be6d0cbc82178c6a => generated 0 bytes in 340 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 04:48:13.095313 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523feca1-f4a6-44be-8fcf-01c2da19a8e4 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:48:13.096749 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-523feca1-f4a6-44be-8fcf-01c2da19a8e4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:48:13.113835 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523feca1-f4a6-44be-8fcf-01c2da19a8e4 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:48:13.174444 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2eb30b-97fc-484d-9890-55569582066a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:48:13.175092 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2eb30b-97fc-484d-9890-55569582066a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:48:13.175374 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2eb30b-97fc-484d-9890-55569582066a admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:48:13.175657 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2eb30b-97fc-484d-9890-55569582066a admin admin] PATH_INFO: `/v3/projects/8029044c500742ab907004dc94b3cb6e` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:48:13.176805 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f2eb30b-97fc-484d-9890-55569582066a admin admin] RBAC: Authorizing `identity:delete_project(project_id=8029044c500742ab907004dc94b3cb6e)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:48:13.178755 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f2eb30b-97fc-484d-9890-55569582066a admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:48:13.300367 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-6f2eb30b-97fc-484d-9890-55569582066a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8029044c500742ab907004dc94b3cb6e', 'request_id': 'req-6f2eb30b-97fc-484d-9890-55569582066a'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:48:13.362145 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-6f2eb30b-97fc-484d-9890-55569582066a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8029044c500742ab907004dc94b3cb6e', 'request_id': 'req-6f2eb30b-97fc-484d-9890-55569582066a'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:48:13.368947 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-6f2eb30b-97fc-484d-9890-55569582066a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8029044c500742ab907004dc94b3cb6e', 'request_id': 'req-6f2eb30b-97fc-484d-9890-55569582066a'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:48:13.383894 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-6f2eb30b-97fc-484d-9890-55569582066a admin admin] The token cache is being invalidate because project 8029044c500742ab907004dc94b3cb6e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81697) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 04:48:13.384185 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-6f2eb30b-97fc-484d-9890-55569582066a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:48:13.384836 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-6f2eb30b-97fc-484d-9890-55569582066a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:48:13.386398 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1327/2668] 10.222.0.30 () {58 vars in 1233 bytes} [Tue Sep 6 04:48:13 2022] DELETE /identity/v3/projects/8029044c500742ab907004dc94b3cb6e => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 04:48:13.391402 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec58e3fb-8f9e-433b-bd41-ad8c43329faf None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:48:13.392562 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec58e3fb-8f9e-433b-bd41-ad8c43329faf None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:48:13.392562 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec58e3fb-8f9e-433b-bd41-ad8c43329faf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:48:13.412074 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-ec58e3fb-8f9e-433b-bd41-ad8c43329faf None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:48:13.413263 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-ec58e3fb-8f9e-433b-bd41-ad8c43329faf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:48:13.463823 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:48:13.477737 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1342/2669] 10.222.0.30 () {58 vars in 958 bytes} [Tue Sep 6 04:48:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 04:48:13.490492 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae81e06f-7b9f-4cfa-81c3-e7062a8b49ed None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:48:13.491167 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae81e06f-7b9f-4cfa-81c3-e7062a8b49ed None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:48:13.491507 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae81e06f-7b9f-4cfa-81c3-e7062a8b49ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:48:13.504555 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:48:13.518221 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-ae81e06f-7b9f-4cfa-81c3-e7062a8b49ed None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:48:13.519882 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-ae81e06f-7b9f-4cfa-81c3-e7062a8b49ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:48:13.530792 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1328/2670] 10.222.0.30 () {58 vars in 958 bytes} [Tue Sep 6 04:48:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 04:48:13.535656 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec58e3fb-8f9e-433b-bd41-ad8c43329faf None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:48:13.547214 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec58e3fb-8f9e-433b-bd41-ad8c43329faf None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:48:13.551894 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9e528b-1d8e-468f-a756-074d669bf290 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:48:13.552556 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9e528b-1d8e-468f-a756-074d669bf290 admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:48:13.552879 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9e528b-1d8e-468f-a756-074d669bf290 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:48:13.553151 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9e528b-1d8e-468f-a756-074d669bf290 admin admin] PATH_INFO: `/v3/domains` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:48:13.553835 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9e528b-1d8e-468f-a756-074d669bf290 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:48:13.554215 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9e528b-1d8e-468f-a756-074d669bf290 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:48:13.555791 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9e528b-1d8e-468f-a756-074d669bf290 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:48:13.568442 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1343/2671] 10.222.0.30 () {58 vars in 1153 bytes} [Tue Sep 6 04:48:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:48:13.601689 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae81e06f-7b9f-4cfa-81c3-e7062a8b49ed None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:48:13.608635 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae81e06f-7b9f-4cfa-81c3-e7062a8b49ed None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:48:13.613496 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77021e6-b6bd-493d-840d-139596ca1414 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:48:13.614048 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77021e6-b6bd-493d-840d-139596ca1414 admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:48:13.614327 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77021e6-b6bd-493d-840d-139596ca1414 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:48:13.614596 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77021e6-b6bd-493d-840d-139596ca1414 admin admin] PATH_INFO: `/v3/projects` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:48:13.615249 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c77021e6-b6bd-493d-840d-139596ca1414 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:48:13.617113 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c77021e6-b6bd-493d-840d-139596ca1414 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:48:13.725493 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1329/2672] 10.222.0.30 () {60 vars in 1153 bytes} [Tue Sep 6 04:48:13 2022] POST /identity/v3/projects => generated 387 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 04:48:13.725621 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9e528b-1d8e-468f-a756-074d669bf290 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:48:13.732265 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9e528b-1d8e-468f-a756-074d669bf290 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:48:13.787285 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e738caa-7fd1-468c-83af-6de34c7eab5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:48:13.787890 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e738caa-7fd1-468c-83af-6de34c7eab5d admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:48:13.788169 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e738caa-7fd1-468c-83af-6de34c7eab5d admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:48:13.788438 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e738caa-7fd1-468c-83af-6de34c7eab5d admin admin] PATH_INFO: `/v3/users` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:48:13.789080 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e738caa-7fd1-468c-83af-6de34c7eab5d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:48:13.791099 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e738caa-7fd1-468c-83af-6de34c7eab5d admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:48:13.864905 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1344/2673] 10.222.0.30 () {60 vars in 1144 bytes} [Tue Sep 6 04:48:13 2022] POST /identity/v3/users => generated 294 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 04:48:13.868956 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77021e6-b6bd-493d-840d-139596ca1414 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:48:13.879886 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77021e6-b6bd-493d-840d-139596ca1414 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:48:13.884524 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0709cfb-2667-47a6-af00-28545888e85e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:48:13.887580 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0709cfb-2667-47a6-af00-28545888e85e admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:48:13.887580 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0709cfb-2667-47a6-af00-28545888e85e admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:48:13.887580 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0709cfb-2667-47a6-af00-28545888e85e admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:48:13.887580 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0709cfb-2667-47a6-af00-28545888e85e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:48:13.888259 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0709cfb-2667-47a6-af00-28545888e85e admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:48:13.898595 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1330/2674] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Sep 6 04:48:13 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 04:48:13.902232 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e738caa-7fd1-468c-83af-6de34c7eab5d admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:48:13.913433 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e738caa-7fd1-468c-83af-6de34c7eab5d admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:48:13.918269 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996e30ca-85ab-45f9-ae52-a702c41e58ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:48:13.919588 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996e30ca-85ab-45f9-ae52-a702c41e58ad admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:48:13.919763 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996e30ca-85ab-45f9-ae52-a702c41e58ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:48:13.920036 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996e30ca-85ab-45f9-ae52-a702c41e58ad admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:48:13.920676 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-996e30ca-85ab-45f9-ae52-a702c41e58ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:48:13.922358 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-996e30ca-85ab-45f9-ae52-a702c41e58ad admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:48:13.933581 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1345/2675] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Sep 6 04:48:13 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 06 04:48:13.937710 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0709cfb-2667-47a6-af00-28545888e85e admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:48:13.947713 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0709cfb-2667-47a6-af00-28545888e85e admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:48:13.952435 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8705ded1-7fbc-43b5-88ac-527318954de9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:48:13.955320 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8705ded1-7fbc-43b5-88ac-527318954de9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:48:13.955320 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8705ded1-7fbc-43b5-88ac-527318954de9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:48:13.955320 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8705ded1-7fbc-43b5-88ac-527318954de9 admin admin] PATH_INFO: `/v3/projects/f23a304a325748acba37d773f3b1fb35/users/95a863a84a724bc68348dd3b5277358d/roles/29607b2de39f40d68bb6ceef64b8a732` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:48:13.968925 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8705ded1-7fbc-43b5-88ac-527318954de9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f23a304a325748acba37d773f3b1fb35, user_id=95a863a84a724bc68348dd3b5277358d, role_id=29607b2de39f40d68bb6ceef64b8a732)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:48:13.971684 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8705ded1-7fbc-43b5-88ac-527318954de9 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:48:14.064736 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1331/2676] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Sep 6 04:48:13 2022] PUT /identity/v3/projects/f23a304a325748acba37d773f3b1fb35/users/95a863a84a724bc68348dd3b5277358d/roles/29607b2de39f40d68bb6ceef64b8a732 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 04:48:14.075059 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c5690f-3389-4648-a47b-e51c6ceae173 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:48:14.076004 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c5690f-3389-4648-a47b-e51c6ceae173 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:48:14.076503 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c5690f-3389-4648-a47b-e51c6ceae173 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:48:14.104603 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-c4c5690f-3389-4648-a47b-e51c6ceae173 None None] MFA Rules not processed for user `95a863a84a724bc68348dd3b5277358d`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:48:14.105890 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-c4c5690f-3389-4648-a47b-e51c6ceae173 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:48:14.169971 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1346/2677] 10.222.0.30 () {58 vars in 958 bytes} [Tue Sep 6 04:48:14 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 04:48:14.174479 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8705ded1-7fbc-43b5-88ac-527318954de9 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:48:14.236450 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8705ded1-7fbc-43b5-88ac-527318954de9 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:48:14.241106 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40970ba-d980-4e0e-921b-2dff7dd43e35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:48:14.241763 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40970ba-d980-4e0e-921b-2dff7dd43e35 admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:48:14.242042 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40970ba-d980-4e0e-921b-2dff7dd43e35 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:48:14.242318 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40970ba-d980-4e0e-921b-2dff7dd43e35 admin admin] PATH_INFO: `/v3/projects` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:48:14.242965 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40970ba-d980-4e0e-921b-2dff7dd43e35 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:48:14.244861 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40970ba-d980-4e0e-921b-2dff7dd43e35 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:48:14.381017 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1332/2678] 10.222.0.30 () {60 vars in 1153 bytes} [Tue Sep 6 04:48:14 2022] POST /identity/v3/projects => generated 389 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 04:48:14.386259 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c5690f-3389-4648-a47b-e51c6ceae173 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:48:14.397877 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c5690f-3389-4648-a47b-e51c6ceae173 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:48:14.447376 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:48:14.459927 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64752392-479e-4672-ab00-de442a35284c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:48:14.460537 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64752392-479e-4672-ab00-de442a35284c admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:48:14.460811 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64752392-479e-4672-ab00-de442a35284c admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:48:14.461081 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64752392-479e-4672-ab00-de442a35284c admin admin] PATH_INFO: `/v3/users` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:48:14.461756 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64752392-479e-4672-ab00-de442a35284c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:48:14.463648 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64752392-479e-4672-ab00-de442a35284c admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:48:14.557229 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1347/2679] 10.222.0.30 () {60 vars in 1144 bytes} [Tue Sep 6 04:48:14 2022] POST /identity/v3/users => generated 301 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 04:48:14.562038 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40970ba-d980-4e0e-921b-2dff7dd43e35 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:48:14.572252 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40970ba-d980-4e0e-921b-2dff7dd43e35 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:48:14.577542 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad72d21f-c71f-47f7-96a0-091ab6a41867 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:48:14.578145 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad72d21f-c71f-47f7-96a0-091ab6a41867 admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:48:14.578427 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad72d21f-c71f-47f7-96a0-091ab6a41867 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:48:14.578699 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad72d21f-c71f-47f7-96a0-091ab6a41867 admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:48:14.579336 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad72d21f-c71f-47f7-96a0-091ab6a41867 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:48:14.580915 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad72d21f-c71f-47f7-96a0-091ab6a41867 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:48:14.591662 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1333/2680] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Sep 6 04:48:14 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 06 04:48:14.595978 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64752392-479e-4672-ab00-de442a35284c admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:48:14.605922 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64752392-479e-4672-ab00-de442a35284c admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:48:14.610931 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5bad71-b264-41bc-9106-7713794fd6ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:48:14.611717 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5bad71-b264-41bc-9106-7713794fd6ad admin admin] REQUEST_METHOD: `PUT` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:48:14.612004 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5bad71-b264-41bc-9106-7713794fd6ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:48:14.612279 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5bad71-b264-41bc-9106-7713794fd6ad admin admin] PATH_INFO: `/v3/projects/2a309fd85d26429382ac3c6471a21352/users/d3c713d4a3ed4a60b2ea56b422c6bfdc/roles/ea4c814713b149108e488de855b025e4` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:48:14.628128 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c5bad71-b264-41bc-9106-7713794fd6ad admin admin] RBAC: Authorizing `identity:create_grant(project_id=2a309fd85d26429382ac3c6471a21352, user_id=d3c713d4a3ed4a60b2ea56b422c6bfdc, role_id=ea4c814713b149108e488de855b025e4)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:48:14.630858 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c5bad71-b264-41bc-9106-7713794fd6ad admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:48:14.704103 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1348/2681] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Sep 6 04:48:14 2022] PUT /identity/v3/projects/2a309fd85d26429382ac3c6471a21352/users/d3c713d4a3ed4a60b2ea56b422c6bfdc/roles/ea4c814713b149108e488de855b025e4 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 04:48:14.708094 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad72d21f-c71f-47f7-96a0-091ab6a41867 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:48:14.718196 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad72d21f-c71f-47f7-96a0-091ab6a41867 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:48:14.773556 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6516d692-823b-4845-8dfa-b5d6248c5647 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:48:14.774165 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6516d692-823b-4845-8dfa-b5d6248c5647 admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:48:14.774445 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6516d692-823b-4845-8dfa-b5d6248c5647 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:48:14.774719 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6516d692-823b-4845-8dfa-b5d6248c5647 admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:48:14.775354 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6516d692-823b-4845-8dfa-b5d6248c5647 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:48:14.776934 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6516d692-823b-4845-8dfa-b5d6248c5647 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:48:14.787468 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1334/2682] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Sep 6 04:48:14 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 04:48:14.791756 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5bad71-b264-41bc-9106-7713794fd6ad admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:48:14.802713 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5bad71-b264-41bc-9106-7713794fd6ad admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:48:14.807427 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6193c54-b3d3-4d42-82ea-ca11211d40ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:48:14.808073 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6193c54-b3d3-4d42-82ea-ca11211d40ac admin admin] REQUEST_METHOD: `PUT` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:48:14.808349 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6193c54-b3d3-4d42-82ea-ca11211d40ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:48:14.808617 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6193c54-b3d3-4d42-82ea-ca11211d40ac admin admin] PATH_INFO: `/v3/projects/2a309fd85d26429382ac3c6471a21352/users/d3c713d4a3ed4a60b2ea56b422c6bfdc/roles/ea4c814713b149108e488de855b025e4` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:48:14.811214 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6193c54-b3d3-4d42-82ea-ca11211d40ac admin admin] RBAC: Authorizing `identity:create_grant(project_id=2a309fd85d26429382ac3c6471a21352, user_id=d3c713d4a3ed4a60b2ea56b422c6bfdc, role_id=ea4c814713b149108e488de855b025e4)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:48:14.820690 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6193c54-b3d3-4d42-82ea-ca11211d40ac admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:48:14.824142 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1349/2683] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Sep 6 04:48:14 2022] PUT /identity/v3/projects/2a309fd85d26429382ac3c6471a21352/users/d3c713d4a3ed4a60b2ea56b422c6bfdc/roles/ea4c814713b149108e488de855b025e4 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 04:48:14.834613 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c5d417-d9a3-451d-a4d5-2d9a8c23243e None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:48:14.834978 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c5d417-d9a3-451d-a4d5-2d9a8c23243e None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:48:14.835264 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c5d417-d9a3-451d-a4d5-2d9a8c23243e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:48:14.855190 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:48:14.863271 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-27c5d417-d9a3-451d-a4d5-2d9a8c23243e None None] MFA Rules not processed for user `d3c713d4a3ed4a60b2ea56b422c6bfdc`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:48:14.864517 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-27c5d417-d9a3-451d-a4d5-2d9a8c23243e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:48:14.934963 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1335/2684] 10.222.0.30 () {58 vars in 958 bytes} [Tue Sep 6 04:48:14 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 04:48:14.946484 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6193c54-b3d3-4d42-82ea-ca11211d40ac admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:48:14.948287 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-f6193c54-b3d3-4d42-82ea-ca11211d40ac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:48:15.008262 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6193c54-b3d3-4d42-82ea-ca11211d40ac admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:48:15.011859 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5149973-f523-4928-b59d-00373fa09293 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd09b5ab675634a2eb4b03755743f7d0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b707733ce65c42edbc1fd95ad0a1c06c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:48:15.012467 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5149973-f523-4928-b59d-00373fa09293 service cinder] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:48:15.012744 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5149973-f523-4928-b59d-00373fa09293 service cinder] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:48:15.013012 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5149973-f523-4928-b59d-00373fa09293 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:48:15.022342 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5149973-f523-4928-b59d-00373fa09293 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:48:15.023794 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5149973-f523-4928-b59d-00373fa09293 service cinder] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:48:15.034181 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1350/2685] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Sep 6 04:48:14 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 04:48:15.170684 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c5d417-d9a3-451d-a4d5-2d9a8c23243e None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:48:15.233849 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c5d417-d9a3-451d-a4d5-2d9a8c23243e None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:48:15.238669 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca815f1-f0e8-434b-869b-2edbcff5d5a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:48:15.239297 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca815f1-f0e8-434b-869b-2edbcff5d5a5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:48:15.239649 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca815f1-f0e8-434b-869b-2edbcff5d5a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:48:15.239919 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca815f1-f0e8-434b-869b-2edbcff5d5a5 admin admin] PATH_INFO: `/v3/users/95a863a84a724bc68348dd3b5277358d` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:48:15.241055 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca815f1-f0e8-434b-869b-2edbcff5d5a5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=95a863a84a724bc68348dd3b5277358d)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:48:15.242968 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca815f1-f0e8-434b-869b-2edbcff5d5a5 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:48:15.408454 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-aca815f1-f0e8-434b-869b-2edbcff5d5a5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '95a863a84a724bc68348dd3b5277358d'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:48:15.474280 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-aca815f1-f0e8-434b-869b-2edbcff5d5a5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '95a863a84a724bc68348dd3b5277358d'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:48:15.474905 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-aca815f1-f0e8-434b-869b-2edbcff5d5a5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '95a863a84a724bc68348dd3b5277358d'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:48:15.482680 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-aca815f1-f0e8-434b-869b-2edbcff5d5a5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '95a863a84a724bc68348dd3b5277358d'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:48:15.495764 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-aca815f1-f0e8-434b-869b-2edbcff5d5a5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '95a863a84a724bc68348dd3b5277358d'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:48:15.501982 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1336/2686] 10.222.0.30 () {58 vars in 1224 bytes} [Tue Sep 6 04:48:15 2022] DELETE /identity/v3/users/95a863a84a724bc68348dd3b5277358d => generated 0 bytes in 332 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 04:48:15.506428 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8724bf6a-eea7-475e-a84c-368f979dc924 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:48:15.506908 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8724bf6a-eea7-475e-a84c-368f979dc924 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:48:15.507011 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8724bf6a-eea7-475e-a84c-368f979dc924 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:48:15.525726 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-8724bf6a-eea7-475e-a84c-368f979dc924 None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:48:15.526898 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-8724bf6a-eea7-475e-a84c-368f979dc924 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:48:15.585985 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1351/2687] 10.222.0.30 () {58 vars in 958 bytes} [Tue Sep 6 04:48:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 04:48:15.604102 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca815f1-f0e8-434b-869b-2edbcff5d5a5 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:48:15.605904 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-aca815f1-f0e8-434b-869b-2edbcff5d5a5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:48:15.625462 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:48:15.647959 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8724bf6a-eea7-475e-a84c-368f979dc924 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:48:15.649365 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-8724bf6a-eea7-475e-a84c-368f979dc924 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:48:15.673857 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca815f1-f0e8-434b-869b-2edbcff5d5a5 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:48:15.677663 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5413de4c-1aec-45cd-b167-16b77a942ac4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90492440f6a6471e8392361c098428aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b707733ce65c42edbc1fd95ad0a1c06c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:48:15.678271 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5413de4c-1aec-45cd-b167-16b77a942ac4 service neutron] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:48:15.678552 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5413de4c-1aec-45cd-b167-16b77a942ac4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:48:15.678827 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5413de4c-1aec-45cd-b167-16b77a942ac4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:48:15.689275 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5413de4c-1aec-45cd-b167-16b77a942ac4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:48:15.690746 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5413de4c-1aec-45cd-b167-16b77a942ac4 service neutron] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:48:15.696761 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:48:15.700385 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1337/2688] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Sep 6 04:48:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 06 04:48:15.709522 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8724bf6a-eea7-475e-a84c-368f979dc924 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:48:15.713157 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3563b86-e041-435a-a0a8-e5cd4531bdbb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90492440f6a6471e8392361c098428aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b707733ce65c42edbc1fd95ad0a1c06c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:48:15.713820 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3563b86-e041-435a-a0a8-e5cd4531bdbb service neutron] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:48:15.714101 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3563b86-e041-435a-a0a8-e5cd4531bdbb service neutron] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:48:15.714388 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3563b86-e041-435a-a0a8-e5cd4531bdbb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:48:15.716474 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-b3563b86-e041-435a-a0a8-e5cd4531bdbb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:48:15.729829 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3563b86-e041-435a-a0a8-e5cd4531bdbb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:48:15.731274 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3563b86-e041-435a-a0a8-e5cd4531bdbb service neutron] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:48:15.742867 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1352/2689] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Sep 6 04:48:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 06 04:48:16.249888 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5413de4c-1aec-45cd-b167-16b77a942ac4 service neutron] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:48:16.251473 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-5413de4c-1aec-45cd-b167-16b77a942ac4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:48:16.267736 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5413de4c-1aec-45cd-b167-16b77a942ac4 service neutron] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:48:16.283057 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17386076-abb3-42d7-b92a-5cd7e54abe07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:48:16.284121 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17386076-abb3-42d7-b92a-5cd7e54abe07 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:48:16.284936 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17386076-abb3-42d7-b92a-5cd7e54abe07 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:48:16.285429 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17386076-abb3-42d7-b92a-5cd7e54abe07 admin admin] PATH_INFO: `/v3/projects/f23a304a325748acba37d773f3b1fb35` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:48:16.287064 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17386076-abb3-42d7-b92a-5cd7e54abe07 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f23a304a325748acba37d773f3b1fb35)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:48:16.289012 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17386076-abb3-42d7-b92a-5cd7e54abe07 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:48:16.394323 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-17386076-abb3-42d7-b92a-5cd7e54abe07 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f23a304a325748acba37d773f3b1fb35', 'request_id': 'req-17386076-abb3-42d7-b92a-5cd7e54abe07'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:48:16.482237 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-17386076-abb3-42d7-b92a-5cd7e54abe07 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f23a304a325748acba37d773f3b1fb35', 'request_id': 'req-17386076-abb3-42d7-b92a-5cd7e54abe07'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:48:16.489595 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-17386076-abb3-42d7-b92a-5cd7e54abe07 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f23a304a325748acba37d773f3b1fb35', 'request_id': 'req-17386076-abb3-42d7-b92a-5cd7e54abe07'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:48:16.502199 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-17386076-abb3-42d7-b92a-5cd7e54abe07 admin admin] The token cache is being invalidate because project f23a304a325748acba37d773f3b1fb35 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81697) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 04:48:16.502424 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-17386076-abb3-42d7-b92a-5cd7e54abe07 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:48:16.503188 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-17386076-abb3-42d7-b92a-5cd7e54abe07 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:48:16.504651 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1338/2690] 10.222.0.30 () {58 vars in 1233 bytes} [Tue Sep 6 04:48:16 2022] DELETE /identity/v3/projects/f23a304a325748acba37d773f3b1fb35 => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 04:48:16.508645 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3563b86-e041-435a-a0a8-e5cd4531bdbb service neutron] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:48:16.519061 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3563b86-e041-435a-a0a8-e5cd4531bdbb service neutron] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:48:16.575671 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99af8127-7b29-4837-acab-24d126a2fe05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:48:16.576344 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99af8127-7b29-4837-acab-24d126a2fe05 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:48:16.577134 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99af8127-7b29-4837-acab-24d126a2fe05 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:48:16.577443 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99af8127-7b29-4837-acab-24d126a2fe05 admin admin] PATH_INFO: `/v3/users/d3c713d4a3ed4a60b2ea56b422c6bfdc` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:48:16.578692 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99af8127-7b29-4837-acab-24d126a2fe05 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d3c713d4a3ed4a60b2ea56b422c6bfdc)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:48:16.580622 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99af8127-7b29-4837-acab-24d126a2fe05 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:48:16.800198 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-99af8127-7b29-4837-acab-24d126a2fe05 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd3c713d4a3ed4a60b2ea56b422c6bfdc'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:48:16.864889 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-99af8127-7b29-4837-acab-24d126a2fe05 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd3c713d4a3ed4a60b2ea56b422c6bfdc'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:48:16.865638 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-99af8127-7b29-4837-acab-24d126a2fe05 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd3c713d4a3ed4a60b2ea56b422c6bfdc'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:48:16.880093 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-99af8127-7b29-4837-acab-24d126a2fe05 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd3c713d4a3ed4a60b2ea56b422c6bfdc'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:48:16.887867 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-99af8127-7b29-4837-acab-24d126a2fe05 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd3c713d4a3ed4a60b2ea56b422c6bfdc'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:48:16.895508 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1353/2691] 10.222.0.30 () {58 vars in 1224 bytes} [Tue Sep 6 04:48:16 2022] DELETE /identity/v3/users/d3c713d4a3ed4a60b2ea56b422c6bfdc => generated 0 bytes in 387 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 04:48:17.284355 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17386076-abb3-42d7-b92a-5cd7e54abe07 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:48:17.286422 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-17386076-abb3-42d7-b92a-5cd7e54abe07 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:48:17.310464 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17386076-abb3-42d7-b92a-5cd7e54abe07 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:48:17.375392 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213cde34-3977-4a19-9be0-6b626e72041e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:48:17.376221 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213cde34-3977-4a19-9be0-6b626e72041e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:48:17.376563 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213cde34-3977-4a19-9be0-6b626e72041e admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:48:17.376967 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213cde34-3977-4a19-9be0-6b626e72041e admin admin] PATH_INFO: `/v3/projects/2a309fd85d26429382ac3c6471a21352` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:48:17.378157 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213cde34-3977-4a19-9be0-6b626e72041e admin admin] RBAC: Authorizing `identity:delete_project(project_id=2a309fd85d26429382ac3c6471a21352)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:48:17.380046 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213cde34-3977-4a19-9be0-6b626e72041e admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:48:17.473206 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:48:17.477238 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-213cde34-3977-4a19-9be0-6b626e72041e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2a309fd85d26429382ac3c6471a21352', 'request_id': 'req-213cde34-3977-4a19-9be0-6b626e72041e'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:48:17.522313 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-213cde34-3977-4a19-9be0-6b626e72041e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2a309fd85d26429382ac3c6471a21352', 'request_id': 'req-213cde34-3977-4a19-9be0-6b626e72041e'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:48:17.530753 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-213cde34-3977-4a19-9be0-6b626e72041e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2a309fd85d26429382ac3c6471a21352', 'request_id': 'req-213cde34-3977-4a19-9be0-6b626e72041e'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:48:17.542954 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-213cde34-3977-4a19-9be0-6b626e72041e admin admin] The token cache is being invalidate because project 2a309fd85d26429382ac3c6471a21352 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81697) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 04:48:17.543247 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-213cde34-3977-4a19-9be0-6b626e72041e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:48:17.543938 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-213cde34-3977-4a19-9be0-6b626e72041e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:48:17.545798 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1339/2692] 10.222.0.30 () {58 vars in 1233 bytes} [Tue Sep 6 04:48:17 2022] DELETE /identity/v3/projects/2a309fd85d26429382ac3c6471a21352 => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 04:48:17.550707 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e759f7-e240-4de9-9616-b926e6d680a7 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:48:17.551207 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e759f7-e240-4de9-9616-b926e6d680a7 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:48:17.551697 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e759f7-e240-4de9-9616-b926e6d680a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:48:17.573077 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-e4e759f7-e240-4de9-9616-b926e6d680a7 None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:48:17.574532 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-e4e759f7-e240-4de9-9616-b926e6d680a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:48:17.643841 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1354/2693] 10.222.0.30 () {58 vars in 958 bytes} [Tue Sep 6 04:48:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 04:48:17.652769 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14930b4-6f7f-4554-ae3e-3fe79f9f991d None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:48:17.653132 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14930b4-6f7f-4554-ae3e-3fe79f9f991d None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:48:17.653515 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14930b4-6f7f-4554-ae3e-3fe79f9f991d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:48:17.675214 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-f14930b4-6f7f-4554-ae3e-3fe79f9f991d None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:48:17.680665 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-f14930b4-6f7f-4554-ae3e-3fe79f9f991d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:48:17.685781 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1340/2694] 10.222.0.30 () {58 vars in 958 bytes} [Tue Sep 6 04:48:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 04:48:17.690984 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e759f7-e240-4de9-9616-b926e6d680a7 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:48:17.701635 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e759f7-e240-4de9-9616-b926e6d680a7 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:48:17.715021 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c3a84e-eb1d-443d-a694-342e9d98a036 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:48:17.715021 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c3a84e-eb1d-443d-a694-342e9d98a036 admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:48:17.715021 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c3a84e-eb1d-443d-a694-342e9d98a036 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:48:17.715021 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c3a84e-eb1d-443d-a694-342e9d98a036 admin admin] PATH_INFO: `/v3/domains` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:48:17.716254 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c3a84e-eb1d-443d-a694-342e9d98a036 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:48:17.716254 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c3a84e-eb1d-443d-a694-342e9d98a036 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:48:17.716254 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c3a84e-eb1d-443d-a694-342e9d98a036 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:48:17.716254 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:48:17.728688 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1355/2695] 10.222.0.30 () {58 vars in 1153 bytes} [Tue Sep 6 04:48:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:48:17.733444 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14930b4-6f7f-4554-ae3e-3fe79f9f991d None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:48:17.743650 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14930b4-6f7f-4554-ae3e-3fe79f9f991d None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:48:17.750322 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f48f861-a5b4-4bf4-8afc-5a4833d42674 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:48:17.750322 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f48f861-a5b4-4bf4-8afc-5a4833d42674 admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:48:17.750322 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f48f861-a5b4-4bf4-8afc-5a4833d42674 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:48:17.750322 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f48f861-a5b4-4bf4-8afc-5a4833d42674 admin admin] PATH_INFO: `/v3/projects` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:48:17.750867 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f48f861-a5b4-4bf4-8afc-5a4833d42674 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:48:17.752759 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f48f861-a5b4-4bf4-8afc-5a4833d42674 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:48:17.851564 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1341/2696] 10.222.0.30 () {60 vars in 1153 bytes} [Tue Sep 6 04:48:17 2022] POST /identity/v3/projects => generated 383 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 04:48:17.855657 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c3a84e-eb1d-443d-a694-342e9d98a036 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:48:17.866234 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c3a84e-eb1d-443d-a694-342e9d98a036 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:48:17.934063 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d50e4f-7ed5-4283-88e2-1d789a58d72f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:48:17.934708 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d50e4f-7ed5-4283-88e2-1d789a58d72f admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:48:17.934984 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d50e4f-7ed5-4283-88e2-1d789a58d72f admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:48:17.935250 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d50e4f-7ed5-4283-88e2-1d789a58d72f admin admin] PATH_INFO: `/v3/users` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:48:17.935941 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d50e4f-7ed5-4283-88e2-1d789a58d72f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:48:17.937950 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d50e4f-7ed5-4283-88e2-1d789a58d72f admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:48:18.020988 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1356/2697] 10.222.0.30 () {60 vars in 1144 bytes} [Tue Sep 6 04:48:17 2022] POST /identity/v3/users => generated 292 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 04:48:18.025225 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f48f861-a5b4-4bf4-8afc-5a4833d42674 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:48:18.036827 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f48f861-a5b4-4bf4-8afc-5a4833d42674 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:48:18.041609 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbaa837-9863-4a5b-81cc-3d926671142c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:48:18.042202 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbaa837-9863-4a5b-81cc-3d926671142c admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:48:18.042477 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbaa837-9863-4a5b-81cc-3d926671142c admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:48:18.042886 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbaa837-9863-4a5b-81cc-3d926671142c admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:48:18.043553 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bbaa837-9863-4a5b-81cc-3d926671142c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:48:18.045126 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bbaa837-9863-4a5b-81cc-3d926671142c admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:48:18.056225 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1342/2698] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Sep 6 04:48:18 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 04:48:18.060141 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d50e4f-7ed5-4283-88e2-1d789a58d72f admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:48:18.070915 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d50e4f-7ed5-4283-88e2-1d789a58d72f admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:48:18.075594 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb65a96-dfed-4002-8d4b-a2da0d5c399b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:48:18.076262 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb65a96-dfed-4002-8d4b-a2da0d5c399b admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:48:18.077145 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb65a96-dfed-4002-8d4b-a2da0d5c399b admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:48:18.077514 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb65a96-dfed-4002-8d4b-a2da0d5c399b admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:48:18.078264 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb65a96-dfed-4002-8d4b-a2da0d5c399b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:48:18.079937 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb65a96-dfed-4002-8d4b-a2da0d5c399b admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:48:18.091054 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1357/2699] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Sep 6 04:48:18 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 04:48:18.096226 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbaa837-9863-4a5b-81cc-3d926671142c admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:48:18.106112 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbaa837-9863-4a5b-81cc-3d926671142c admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:48:18.111222 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4dd58f6-544c-4c1c-af2d-57692e9e15c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:48:18.111877 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4dd58f6-544c-4c1c-af2d-57692e9e15c3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:48:18.112156 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4dd58f6-544c-4c1c-af2d-57692e9e15c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:48:18.112427 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4dd58f6-544c-4c1c-af2d-57692e9e15c3 admin admin] PATH_INFO: `/v3/projects/ff780898725f4e5fae5098f2eb31ccb5/users/a4ad8c8211d040b5bdb7e5deabdb76ca/roles/29607b2de39f40d68bb6ceef64b8a732` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:48:18.127157 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4dd58f6-544c-4c1c-af2d-57692e9e15c3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ff780898725f4e5fae5098f2eb31ccb5, user_id=a4ad8c8211d040b5bdb7e5deabdb76ca, role_id=29607b2de39f40d68bb6ceef64b8a732)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:48:18.129806 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4dd58f6-544c-4c1c-af2d-57692e9e15c3 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:48:18.192805 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1343/2700] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Sep 6 04:48:18 2022] PUT /identity/v3/projects/ff780898725f4e5fae5098f2eb31ccb5/users/a4ad8c8211d040b5bdb7e5deabdb76ca/roles/29607b2de39f40d68bb6ceef64b8a732 => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 04:48:18.204021 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5ed520-2320-4d47-900f-6f69cfbe6f48 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:48:18.204483 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5ed520-2320-4d47-900f-6f69cfbe6f48 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:48:18.204845 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5ed520-2320-4d47-900f-6f69cfbe6f48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:48:18.233763 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-cc5ed520-2320-4d47-900f-6f69cfbe6f48 None None] MFA Rules not processed for user `a4ad8c8211d040b5bdb7e5deabdb76ca`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:48:18.235191 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-cc5ed520-2320-4d47-900f-6f69cfbe6f48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:48:18.298956 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1358/2701] 10.222.0.30 () {58 vars in 958 bytes} [Tue Sep 6 04:48:18 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 04:48:18.310199 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4dd58f6-544c-4c1c-af2d-57692e9e15c3 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:48:18.312624 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-d4dd58f6-544c-4c1c-af2d-57692e9e15c3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:48:18.397231 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4dd58f6-544c-4c1c-af2d-57692e9e15c3 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:48:18.397231 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7497eee2-0b60-46a9-8669-0ad4d94748d8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd09b5ab675634a2eb4b03755743f7d0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b707733ce65c42edbc1fd95ad0a1c06c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:48:18.397231 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7497eee2-0b60-46a9-8669-0ad4d94748d8 service cinder] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:48:18.397231 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7497eee2-0b60-46a9-8669-0ad4d94748d8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:48:18.398492 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7497eee2-0b60-46a9-8669-0ad4d94748d8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:48:18.399629 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7497eee2-0b60-46a9-8669-0ad4d94748d8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:48:18.401132 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7497eee2-0b60-46a9-8669-0ad4d94748d8 service cinder] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:48:18.413136 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1344/2702] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Sep 6 04:48:18 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 04:48:18.439211 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5ed520-2320-4d47-900f-6f69cfbe6f48 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:48:18.508490 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5ed520-2320-4d47-900f-6f69cfbe6f48 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:48:18.513271 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e1417a-7950-4d79-9b54-54510a4772cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:48:18.513974 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e1417a-7950-4d79-9b54-54510a4772cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:48:18.514313 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e1417a-7950-4d79-9b54-54510a4772cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:48:18.514585 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e1417a-7950-4d79-9b54-54510a4772cc admin admin] PATH_INFO: `/v3/users/a4ad8c8211d040b5bdb7e5deabdb76ca` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:48:18.515729 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e1417a-7950-4d79-9b54-54510a4772cc admin admin] RBAC: Authorizing `identity:delete_user(user_id=a4ad8c8211d040b5bdb7e5deabdb76ca)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:48:18.517726 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e1417a-7950-4d79-9b54-54510a4772cc admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:48:18.520156 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:48:18.700007 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-b4e1417a-7950-4d79-9b54-54510a4772cc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a4ad8c8211d040b5bdb7e5deabdb76ca'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:48:18.800144 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-b4e1417a-7950-4d79-9b54-54510a4772cc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a4ad8c8211d040b5bdb7e5deabdb76ca'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:48:18.800789 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-b4e1417a-7950-4d79-9b54-54510a4772cc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a4ad8c8211d040b5bdb7e5deabdb76ca'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:48:18.814624 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-b4e1417a-7950-4d79-9b54-54510a4772cc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a4ad8c8211d040b5bdb7e5deabdb76ca'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:48:18.829662 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-b4e1417a-7950-4d79-9b54-54510a4772cc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a4ad8c8211d040b5bdb7e5deabdb76ca'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:48:18.837925 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1359/2703] 10.222.0.30 () {58 vars in 1224 bytes} [Tue Sep 6 04:48:18 2022] DELETE /identity/v3/users/a4ad8c8211d040b5bdb7e5deabdb76ca => generated 0 bytes in 399 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 04:48:18.843332 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cbc71e-27f3-4524-ba34-b235607468f2 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:48:18.843630 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cbc71e-27f3-4524-ba34-b235607468f2 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:48:18.843901 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cbc71e-27f3-4524-ba34-b235607468f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:48:18.871035 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-e3cbc71e-27f3-4524-ba34-b235607468f2 None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:48:18.872253 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-e3cbc71e-27f3-4524-ba34-b235607468f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:48:18.965992 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1345/2704] 10.222.0.30 () {58 vars in 958 bytes} [Tue Sep 6 04:48:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 04:48:18.977815 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e1417a-7950-4d79-9b54-54510a4772cc admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:48:18.979260 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-b4e1417a-7950-4d79-9b54-54510a4772cc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:48:19.076156 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e1417a-7950-4d79-9b54-54510a4772cc admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:48:19.080364 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e80afd-82d0-4805-8974-64ee508a782c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90492440f6a6471e8392361c098428aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b707733ce65c42edbc1fd95ad0a1c06c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:48:19.081130 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e80afd-82d0-4805-8974-64ee508a782c service neutron] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:48:19.081569 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e80afd-82d0-4805-8974-64ee508a782c service neutron] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:48:19.081949 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e80afd-82d0-4805-8974-64ee508a782c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:48:19.095770 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e80afd-82d0-4805-8974-64ee508a782c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:48:19.097324 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e80afd-82d0-4805-8974-64ee508a782c service neutron] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:48:19.109625 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1360/2705] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Sep 6 04:48:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 04:48:19.539145 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3cbc71e-27f3-4524-ba34-b235607468f2 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:48:19.539145 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-e3cbc71e-27f3-4524-ba34-b235607468f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:48:19.551759 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3cbc71e-27f3-4524-ba34-b235607468f2 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:48:19.563032 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2defc3fe-cb72-4868-ad81-ed45eef57792 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:48:19.567496 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2defc3fe-cb72-4868-ad81-ed45eef57792 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:48:19.567496 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2defc3fe-cb72-4868-ad81-ed45eef57792 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:48:19.567496 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2defc3fe-cb72-4868-ad81-ed45eef57792 admin admin] PATH_INFO: `/v3/projects/ff780898725f4e5fae5098f2eb31ccb5` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:48:19.567496 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2defc3fe-cb72-4868-ad81-ed45eef57792 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ff780898725f4e5fae5098f2eb31ccb5)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:48:19.567496 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2defc3fe-cb72-4868-ad81-ed45eef57792 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:48:19.575111 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=93,keystone:DELETE=4,keystone:INSERT=3 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:48:19.683728 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-2defc3fe-cb72-4868-ad81-ed45eef57792 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ff780898725f4e5fae5098f2eb31ccb5', 'request_id': 'req-2defc3fe-cb72-4868-ad81-ed45eef57792'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:48:19.731996 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e80afd-82d0-4805-8974-64ee508a782c service neutron] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:48:19.733608 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-89e80afd-82d0-4805-8974-64ee508a782c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:48:19.751675 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e80afd-82d0-4805-8974-64ee508a782c service neutron] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:48:19.777454 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-2defc3fe-cb72-4868-ad81-ed45eef57792 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ff780898725f4e5fae5098f2eb31ccb5', 'request_id': 'req-2defc3fe-cb72-4868-ad81-ed45eef57792'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:48:19.791996 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-2defc3fe-cb72-4868-ad81-ed45eef57792 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ff780898725f4e5fae5098f2eb31ccb5', 'request_id': 'req-2defc3fe-cb72-4868-ad81-ed45eef57792'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:48:19.799425 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-2defc3fe-cb72-4868-ad81-ed45eef57792 admin admin] The token cache is being invalidate because project ff780898725f4e5fae5098f2eb31ccb5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81697) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 04:48:19.799800 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-2defc3fe-cb72-4868-ad81-ed45eef57792 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:48:19.800553 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-2defc3fe-cb72-4868-ad81-ed45eef57792 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:48:19.802467 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1346/2706] 10.222.0.30 () {58 vars in 1233 bytes} [Tue Sep 6 04:48:19 2022] DELETE /identity/v3/projects/ff780898725f4e5fae5098f2eb31ccb5 => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 04:48:19.807135 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c098d8c3-05df-4ebb-a745-759a112d3b7b None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:48:19.807498 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c098d8c3-05df-4ebb-a745-759a112d3b7b None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:48:19.807842 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c098d8c3-05df-4ebb-a745-759a112d3b7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:48:19.817842 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e974ff-bc71-4b7c-8901-054bf60c9ffa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:48:19.820643 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e974ff-bc71-4b7c-8901-054bf60c9ffa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:48:19.820643 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e974ff-bc71-4b7c-8901-054bf60c9ffa admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:48:19.820643 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e974ff-bc71-4b7c-8901-054bf60c9ffa admin admin] PATH_INFO: `/v3/users/1ca147cfc97149dd87e533123a262cfd` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:48:19.820643 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e974ff-bc71-4b7c-8901-054bf60c9ffa admin admin] RBAC: Authorizing `identity:delete_user(user_id=1ca147cfc97149dd87e533123a262cfd)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:48:19.822419 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e974ff-bc71-4b7c-8901-054bf60c9ffa admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:48:19.830283 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-c098d8c3-05df-4ebb-a745-759a112d3b7b None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:48:19.831496 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-c098d8c3-05df-4ebb-a745-759a112d3b7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:48:19.839862 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1347/2707] 10.222.0.30 () {58 vars in 958 bytes} [Tue Sep 6 04:48:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 04:48:19.847679 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256fd153-23db-40c1-8b5f-c820fe2476ff None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:48:19.848152 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256fd153-23db-40c1-8b5f-c820fe2476ff None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:48:19.848430 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256fd153-23db-40c1-8b5f-c820fe2476ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:48:19.869284 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-256fd153-23db-40c1-8b5f-c820fe2476ff None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:48:19.870549 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-256fd153-23db-40c1-8b5f-c820fe2476ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:48:19.879055 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1348/2708] 10.222.0.30 () {58 vars in 958 bytes} [Tue Sep 6 04:48:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 04:48:19.883127 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256fd153-23db-40c1-8b5f-c820fe2476ff None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:48:19.894841 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256fd153-23db-40c1-8b5f-c820fe2476ff None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:48:19.899393 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767a6a10-04c0-4876-adcf-6704fc610ebc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:48:19.899972 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767a6a10-04c0-4876-adcf-6704fc610ebc admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:48:19.900246 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767a6a10-04c0-4876-adcf-6704fc610ebc admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:48:19.900540 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767a6a10-04c0-4876-adcf-6704fc610ebc admin admin] PATH_INFO: `/v3/domains` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:48:19.901171 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767a6a10-04c0-4876-adcf-6704fc610ebc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:48:19.901498 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767a6a10-04c0-4876-adcf-6704fc610ebc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:48:19.903060 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767a6a10-04c0-4876-adcf-6704fc610ebc admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:48:19.911407 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:48:19.915035 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1349/2709] 10.222.0.30 () {58 vars in 1153 bytes} [Tue Sep 6 04:48:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 04:48:19.919287 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767a6a10-04c0-4876-adcf-6704fc610ebc admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:48:19.928844 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767a6a10-04c0-4876-adcf-6704fc610ebc admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:48:19.933359 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49eb3a28-ffc5-4051-aef7-988117e1820a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:48:19.933929 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49eb3a28-ffc5-4051-aef7-988117e1820a admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:48:19.934203 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49eb3a28-ffc5-4051-aef7-988117e1820a admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:48:19.934471 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49eb3a28-ffc5-4051-aef7-988117e1820a admin admin] PATH_INFO: `/v3/projects` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:48:19.935066 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49eb3a28-ffc5-4051-aef7-988117e1820a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:48:19.937485 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49eb3a28-ffc5-4051-aef7-988117e1820a admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:48:19.992509 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-42e974ff-bc71-4b7c-8901-054bf60c9ffa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1ca147cfc97149dd87e533123a262cfd'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:48:20.020891 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1350/2710] 10.222.0.30 () {60 vars in 1153 bytes} [Tue Sep 6 04:48:19 2022] POST /identity/v3/projects => generated 373 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 04:48:20.025024 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49eb3a28-ffc5-4051-aef7-988117e1820a admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:48:20.053049 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-42e974ff-bc71-4b7c-8901-054bf60c9ffa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1ca147cfc97149dd87e533123a262cfd'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:48:20.053694 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-42e974ff-bc71-4b7c-8901-054bf60c9ffa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1ca147cfc97149dd87e533123a262cfd'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:48:20.068326 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-42e974ff-bc71-4b7c-8901-054bf60c9ffa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1ca147cfc97149dd87e533123a262cfd'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:48:20.075408 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-42e974ff-bc71-4b7c-8901-054bf60c9ffa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1ca147cfc97149dd87e533123a262cfd'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:48:20.081655 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1361/2711] 10.222.0.30 () {58 vars in 1224 bytes} [Tue Sep 6 04:48:19 2022] DELETE /identity/v3/users/1ca147cfc97149dd87e533123a262cfd => generated 0 bytes in 350 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 04:48:20.089733 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49eb3a28-ffc5-4051-aef7-988117e1820a admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:48:20.138691 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:48:20.147534 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dcdce31-b6ee-4d17-afea-cf3e569d2d69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:48:20.148150 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcdce31-b6ee-4d17-afea-cf3e569d2d69 admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:48:20.149191 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcdce31-b6ee-4d17-afea-cf3e569d2d69 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:48:20.149191 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcdce31-b6ee-4d17-afea-cf3e569d2d69 admin admin] PATH_INFO: `/v3/users` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:48:20.149420 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dcdce31-b6ee-4d17-afea-cf3e569d2d69 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:48:20.151220 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dcdce31-b6ee-4d17-afea-cf3e569d2d69 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:48:20.225206 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1351/2712] 10.222.0.30 () {60 vars in 1144 bytes} [Tue Sep 6 04:48:20 2022] POST /identity/v3/users => generated 287 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 04:48:20.229269 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e974ff-bc71-4b7c-8901-054bf60c9ffa admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:48:20.230781 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-42e974ff-bc71-4b7c-8901-054bf60c9ffa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:48:20.244150 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e974ff-bc71-4b7c-8901-054bf60c9ffa admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:48:20.249458 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c81387e-72b3-4d9c-922b-62cfdfa6ffec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:48:20.252498 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c81387e-72b3-4d9c-922b-62cfdfa6ffec admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:48:20.252498 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c81387e-72b3-4d9c-922b-62cfdfa6ffec admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:48:20.252498 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c81387e-72b3-4d9c-922b-62cfdfa6ffec admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:48:20.252498 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c81387e-72b3-4d9c-922b-62cfdfa6ffec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:48:20.252868 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c81387e-72b3-4d9c-922b-62cfdfa6ffec admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:48:20.262713 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1362/2713] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Sep 6 04:48:20 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 04:48:20.266942 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dcdce31-b6ee-4d17-afea-cf3e569d2d69 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:48:20.277044 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dcdce31-b6ee-4d17-afea-cf3e569d2d69 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:48:20.282071 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d27214-ce42-496f-83e0-6d6dea20d9d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:48:20.282680 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d27214-ce42-496f-83e0-6d6dea20d9d9 admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:48:20.282961 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d27214-ce42-496f-83e0-6d6dea20d9d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:48:20.283238 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d27214-ce42-496f-83e0-6d6dea20d9d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:48:20.283877 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d27214-ce42-496f-83e0-6d6dea20d9d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:48:20.285530 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d27214-ce42-496f-83e0-6d6dea20d9d9 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:48:20.296348 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1352/2714] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Sep 6 04:48:20 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 04:48:20.300650 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c81387e-72b3-4d9c-922b-62cfdfa6ffec admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:48:20.306986 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d27214-ce42-496f-83e0-6d6dea20d9d9 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:48:20.308753 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-c1d27214-ce42-496f-83e0-6d6dea20d9d9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:48:20.311843 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c81387e-72b3-4d9c-922b-62cfdfa6ffec admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:48:20.317180 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9eedb25-4352-4c99-bdb0-bf5845d47912 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:48:20.317887 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9eedb25-4352-4c99-bdb0-bf5845d47912 admin admin] REQUEST_METHOD: `PUT` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:48:20.318148 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9eedb25-4352-4c99-bdb0-bf5845d47912 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:48:20.318483 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9eedb25-4352-4c99-bdb0-bf5845d47912 admin admin] PATH_INFO: `/v3/projects/581650ced13b4553b05dfb09b851da45/users/c8c5e9aad7fa43b6a13404e0859a1f35/roles/29607b2de39f40d68bb6ceef64b8a732` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:48:20.323231 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d27214-ce42-496f-83e0-6d6dea20d9d9 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:48:20.330177 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777a45a9-5395-449a-af5c-3d2df11111b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:48:20.330831 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777a45a9-5395-449a-af5c-3d2df11111b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:48:20.331114 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777a45a9-5395-449a-af5c-3d2df11111b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:48:20.331501 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777a45a9-5395-449a-af5c-3d2df11111b0 admin admin] PATH_INFO: `/v3/projects/2388589a34d346f3b31b231a9dafe34d` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:48:20.332563 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777a45a9-5395-449a-af5c-3d2df11111b0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2388589a34d346f3b31b231a9dafe34d)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:48:20.334487 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777a45a9-5395-449a-af5c-3d2df11111b0 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:48:20.344674 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9eedb25-4352-4c99-bdb0-bf5845d47912 admin admin] RBAC: Authorizing `identity:create_grant(project_id=581650ced13b4553b05dfb09b851da45, user_id=c8c5e9aad7fa43b6a13404e0859a1f35, role_id=29607b2de39f40d68bb6ceef64b8a732)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:48:20.346773 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9eedb25-4352-4c99-bdb0-bf5845d47912 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:48:20.419097 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1363/2715] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Sep 6 04:48:20 2022] PUT /identity/v3/projects/581650ced13b4553b05dfb09b851da45/users/c8c5e9aad7fa43b6a13404e0859a1f35/roles/29607b2de39f40d68bb6ceef64b8a732 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 04:48:20.454191 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-777a45a9-5395-449a-af5c-3d2df11111b0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2388589a34d346f3b31b231a9dafe34d', 'request_id': 'req-777a45a9-5395-449a-af5c-3d2df11111b0'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:48:20.486045 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e0a207-ea30-48f3-a1dd-ddaad32ba4b1 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:48:20.486337 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e0a207-ea30-48f3-a1dd-ddaad32ba4b1 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:48:20.486633 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e0a207-ea30-48f3-a1dd-ddaad32ba4b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:48:20.514201 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-31e0a207-ea30-48f3-a1dd-ddaad32ba4b1 None None] MFA Rules not processed for user `c8c5e9aad7fa43b6a13404e0859a1f35`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:48:20.515550 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-31e0a207-ea30-48f3-a1dd-ddaad32ba4b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:48:20.537537 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-777a45a9-5395-449a-af5c-3d2df11111b0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2388589a34d346f3b31b231a9dafe34d', 'request_id': 'req-777a45a9-5395-449a-af5c-3d2df11111b0'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:48:20.544213 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-777a45a9-5395-449a-af5c-3d2df11111b0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2388589a34d346f3b31b231a9dafe34d', 'request_id': 'req-777a45a9-5395-449a-af5c-3d2df11111b0'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:48:20.555646 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-777a45a9-5395-449a-af5c-3d2df11111b0 admin admin] The token cache is being invalidate because project 2388589a34d346f3b31b231a9dafe34d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81697) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 04:48:20.555873 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-777a45a9-5395-449a-af5c-3d2df11111b0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:48:20.556528 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-777a45a9-5395-449a-af5c-3d2df11111b0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:48:20.558040 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1353/2716] 10.222.0.30 () {58 vars in 1233 bytes} [Tue Sep 6 04:48:20 2022] DELETE /identity/v3/projects/2388589a34d346f3b31b231a9dafe34d => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 04:48:20.563039 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc804e4d-979c-43df-b57e-036f6dd5515a None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:48:20.563345 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc804e4d-979c-43df-b57e-036f6dd5515a None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:48:20.563617 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc804e4d-979c-43df-b57e-036f6dd5515a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:48:20.580790 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1364/2717] 10.222.0.30 () {58 vars in 958 bytes} [Tue Sep 6 04:48:20 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 04:48:20.583179 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-dc804e4d-979c-43df-b57e-036f6dd5515a None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:48:20.584482 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-dc804e4d-979c-43df-b57e-036f6dd5515a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:48:20.585109 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e0a207-ea30-48f3-a1dd-ddaad32ba4b1 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:48:20.586396 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-31e0a207-ea30-48f3-a1dd-ddaad32ba4b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:48:20.644229 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1354/2718] 10.222.0.30 () {58 vars in 958 bytes} [Tue Sep 6 04:48:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 04:48:20.648958 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:DELETE=6,keystone:SELECT=92,keystone:INSERT=2 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:48:20.652991 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e0a207-ea30-48f3-a1dd-ddaad32ba4b1 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:48:20.653158 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7c8465-59f8-4445-834d-2f6694590c6e None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:48:20.653331 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7c8465-59f8-4445-834d-2f6694590c6e None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:48:20.653717 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7c8465-59f8-4445-834d-2f6694590c6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:48:20.659355 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b893a3-9d1a-406f-a626-9cf7b4128f16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:48:20.659966 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b893a3-9d1a-406f-a626-9cf7b4128f16 admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:48:20.662041 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b893a3-9d1a-406f-a626-9cf7b4128f16 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:48:20.662041 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b893a3-9d1a-406f-a626-9cf7b4128f16 admin admin] PATH_INFO: `/v3/projects` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:48:20.662041 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b893a3-9d1a-406f-a626-9cf7b4128f16 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:48:20.663116 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b893a3-9d1a-406f-a626-9cf7b4128f16 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:48:20.673947 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-7d7c8465-59f8-4445-834d-2f6694590c6e None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:48:20.675148 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-7d7c8465-59f8-4445-834d-2f6694590c6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:48:20.683199 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1355/2719] 10.222.0.30 () {58 vars in 958 bytes} [Tue Sep 6 04:48:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 04:48:20.687825 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7c8465-59f8-4445-834d-2f6694590c6e None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:48:20.696429 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7c8465-59f8-4445-834d-2f6694590c6e None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:48:20.700972 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652cd907-8c16-4f81-9158-fa9688ec9688 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:48:20.701585 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652cd907-8c16-4f81-9158-fa9688ec9688 admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:48:20.702022 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652cd907-8c16-4f81-9158-fa9688ec9688 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:48:20.702395 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652cd907-8c16-4f81-9158-fa9688ec9688 admin admin] PATH_INFO: `/v3/domains` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:48:20.703081 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652cd907-8c16-4f81-9158-fa9688ec9688 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:48:20.703712 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652cd907-8c16-4f81-9158-fa9688ec9688 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:48:20.705333 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652cd907-8c16-4f81-9158-fa9688ec9688 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:48:20.716581 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1356/2720] 10.222.0.30 () {58 vars in 1153 bytes} [Tue Sep 6 04:48:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 04:48:20.721170 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652cd907-8c16-4f81-9158-fa9688ec9688 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:48:20.730713 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652cd907-8c16-4f81-9158-fa9688ec9688 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:48:20.736203 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c733c4-e542-4284-a160-bff699c17ffe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:48:20.736776 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c733c4-e542-4284-a160-bff699c17ffe admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:48:20.737051 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c733c4-e542-4284-a160-bff699c17ffe admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:48:20.737344 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c733c4-e542-4284-a160-bff699c17ffe admin admin] PATH_INFO: `/v3/projects` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:48:20.738109 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c733c4-e542-4284-a160-bff699c17ffe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:48:20.739862 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c733c4-e542-4284-a160-bff699c17ffe admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:48:20.798456 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1365/2721] 10.222.0.30 () {60 vars in 1153 bytes} [Tue Sep 6 04:48:20 2022] POST /identity/v3/projects => generated 373 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 04:48:20.802410 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b893a3-9d1a-406f-a626-9cf7b4128f16 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:48:20.867298 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b893a3-9d1a-406f-a626-9cf7b4128f16 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:48:20.871932 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c048e063-bb90-4c15-afa0-1062386448d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:48:20.872538 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c048e063-bb90-4c15-afa0-1062386448d8 admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:48:20.872818 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c048e063-bb90-4c15-afa0-1062386448d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:48:20.873162 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c048e063-bb90-4c15-afa0-1062386448d8 admin admin] PATH_INFO: `/v3/users` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:48:20.873966 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c048e063-bb90-4c15-afa0-1062386448d8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:48:20.875994 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c048e063-bb90-4c15-afa0-1062386448d8 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:48:20.921526 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1357/2722] 10.222.0.30 () {60 vars in 1153 bytes} [Tue Sep 6 04:48:20 2022] POST /identity/v3/projects => generated 377 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 04:48:20.925545 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c733c4-e542-4284-a160-bff699c17ffe admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:48:20.938455 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:48:20.986970 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c733c4-e542-4284-a160-bff699c17ffe admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:48:20.991554 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6279d41-ecc5-41d0-a289-b221ffc5ef97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:48:20.992149 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6279d41-ecc5-41d0-a289-b221ffc5ef97 admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:48:20.992427 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6279d41-ecc5-41d0-a289-b221ffc5ef97 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:48:20.992788 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6279d41-ecc5-41d0-a289-b221ffc5ef97 admin admin] PATH_INFO: `/v3/users` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:48:20.993445 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6279d41-ecc5-41d0-a289-b221ffc5ef97 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:48:20.995329 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6279d41-ecc5-41d0-a289-b221ffc5ef97 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:48:21.011909 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1366/2723] 10.222.0.30 () {60 vars in 1144 bytes} [Tue Sep 6 04:48:20 2022] POST /identity/v3/users => generated 293 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 04:48:21.016346 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c048e063-bb90-4c15-afa0-1062386448d8 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:48:21.025944 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c048e063-bb90-4c15-afa0-1062386448d8 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:48:21.031437 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c212c9-0e0a-4715-b4e3-a82f4910629e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:48:21.032017 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c212c9-0e0a-4715-b4e3-a82f4910629e admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:48:21.032298 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c212c9-0e0a-4715-b4e3-a82f4910629e admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:48:21.032569 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c212c9-0e0a-4715-b4e3-a82f4910629e admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:48:21.033198 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c212c9-0e0a-4715-b4e3-a82f4910629e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:48:21.034804 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c212c9-0e0a-4715-b4e3-a82f4910629e admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:48:21.044465 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1367/2724] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Sep 6 04:48:21 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 06 04:48:21.048606 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c212c9-0e0a-4715-b4e3-a82f4910629e admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:48:21.057732 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c212c9-0e0a-4715-b4e3-a82f4910629e admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:48:21.062531 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39fd230-973f-40c7-84c5-763810aff70f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:48:21.063136 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39fd230-973f-40c7-84c5-763810aff70f admin admin] REQUEST_METHOD: `PUT` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:48:21.063416 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39fd230-973f-40c7-84c5-763810aff70f admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:48:21.063690 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39fd230-973f-40c7-84c5-763810aff70f admin admin] PATH_INFO: `/v3/projects/e253814312db4066839c6c812d05a16c/users/4618d2beb155455fb225613df6a548e2/roles/ea4c814713b149108e488de855b025e4` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:48:21.074394 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1358/2725] 10.222.0.30 () {60 vars in 1144 bytes} [Tue Sep 6 04:48:20 2022] POST /identity/v3/users => generated 289 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 04:48:21.079123 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6279d41-ecc5-41d0-a289-b221ffc5ef97 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:48:21.083505 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39fd230-973f-40c7-84c5-763810aff70f admin admin] RBAC: Authorizing `identity:create_grant(project_id=e253814312db4066839c6c812d05a16c, user_id=4618d2beb155455fb225613df6a548e2, role_id=ea4c814713b149108e488de855b025e4)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:48:21.083505 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39fd230-973f-40c7-84c5-763810aff70f admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:48:21.089541 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6279d41-ecc5-41d0-a289-b221ffc5ef97 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:48:21.094946 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4894a81-4dbd-40af-bcba-1c7fcb435565 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:48:21.095604 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4894a81-4dbd-40af-bcba-1c7fcb435565 admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:48:21.095887 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4894a81-4dbd-40af-bcba-1c7fcb435565 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:48:21.096213 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4894a81-4dbd-40af-bcba-1c7fcb435565 admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:48:21.096849 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4894a81-4dbd-40af-bcba-1c7fcb435565 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:48:21.098466 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4894a81-4dbd-40af-bcba-1c7fcb435565 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:48:21.108258 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1359/2726] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Sep 6 04:48:21 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 06 04:48:21.112069 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4894a81-4dbd-40af-bcba-1c7fcb435565 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:48:21.121528 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4894a81-4dbd-40af-bcba-1c7fcb435565 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:48:21.127414 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17292f92-0e2d-4107-9049-4406bc333ee6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:48:21.127981 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17292f92-0e2d-4107-9049-4406bc333ee6 admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:48:21.128262 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17292f92-0e2d-4107-9049-4406bc333ee6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:48:21.128536 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17292f92-0e2d-4107-9049-4406bc333ee6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:48:21.129182 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17292f92-0e2d-4107-9049-4406bc333ee6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:48:21.130762 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17292f92-0e2d-4107-9049-4406bc333ee6 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:48:21.141647 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1360/2727] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Sep 6 04:48:21 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 06 04:48:21.145592 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17292f92-0e2d-4107-9049-4406bc333ee6 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:48:21.154766 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17292f92-0e2d-4107-9049-4406bc333ee6 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:48:21.160928 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1368/2728] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Sep 6 04:48:21 2022] PUT /identity/v3/projects/e253814312db4066839c6c812d05a16c/users/4618d2beb155455fb225613df6a548e2/roles/ea4c814713b149108e488de855b025e4 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 04:48:21.164617 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39fd230-973f-40c7-84c5-763810aff70f admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:48:21.215067 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6e3994-1bf0-490d-8a2e-c8e58f98e78d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:48:21.215879 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6e3994-1bf0-490d-8a2e-c8e58f98e78d admin admin] REQUEST_METHOD: `PUT` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:48:21.216161 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6e3994-1bf0-490d-8a2e-c8e58f98e78d admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:48:21.216516 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6e3994-1bf0-490d-8a2e-c8e58f98e78d admin admin] PATH_INFO: `/v3/projects/f91efa94deff4ceb80b0a1aa88087421/users/31b919a031504f2bace596da457b91cf/roles/29607b2de39f40d68bb6ceef64b8a732` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:48:21.236410 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae6e3994-1bf0-490d-8a2e-c8e58f98e78d admin admin] RBAC: Authorizing `identity:create_grant(project_id=f91efa94deff4ceb80b0a1aa88087421, user_id=31b919a031504f2bace596da457b91cf, role_id=29607b2de39f40d68bb6ceef64b8a732)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:48:21.239095 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae6e3994-1bf0-490d-8a2e-c8e58f98e78d admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:48:21.240569 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39fd230-973f-40c7-84c5-763810aff70f admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:48:21.245329 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54420e63-a525-4984-b02b-e011ac273173 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:48:21.245954 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54420e63-a525-4984-b02b-e011ac273173 admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:48:21.246229 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54420e63-a525-4984-b02b-e011ac273173 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:48:21.246495 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54420e63-a525-4984-b02b-e011ac273173 admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:48:21.247162 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54420e63-a525-4984-b02b-e011ac273173 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:48:21.249337 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54420e63-a525-4984-b02b-e011ac273173 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:48:21.259735 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1369/2729] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Sep 6 04:48:21 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 06 04:48:21.263877 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54420e63-a525-4984-b02b-e011ac273173 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:48:21.274156 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54420e63-a525-4984-b02b-e011ac273173 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:48:21.278806 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6622ece5-746d-40b9-8fca-347032259bc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:48:21.279506 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6622ece5-746d-40b9-8fca-347032259bc1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:48:21.279779 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6622ece5-746d-40b9-8fca-347032259bc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:48:21.280044 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6622ece5-746d-40b9-8fca-347032259bc1 admin admin] PATH_INFO: `/v3/projects/e253814312db4066839c6c812d05a16c/users/4618d2beb155455fb225613df6a548e2/roles/ea4c814713b149108e488de855b025e4` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:48:21.282230 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6622ece5-746d-40b9-8fca-347032259bc1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e253814312db4066839c6c812d05a16c, user_id=4618d2beb155455fb225613df6a548e2, role_id=ea4c814713b149108e488de855b025e4)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:48:21.284905 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6622ece5-746d-40b9-8fca-347032259bc1 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:48:21.294750 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1370/2730] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Sep 6 04:48:21 2022] PUT /identity/v3/projects/e253814312db4066839c6c812d05a16c/users/4618d2beb155455fb225613df6a548e2/roles/ea4c814713b149108e488de855b025e4 => generated 0 bytes in 31 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 04:48:21.303569 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1361/2731] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Sep 6 04:48:21 2022] PUT /identity/v3/projects/f91efa94deff4ceb80b0a1aa88087421/users/31b919a031504f2bace596da457b91cf/roles/29607b2de39f40d68bb6ceef64b8a732 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 04:48:21.304907 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510bc4b9-8bd8-49b8-914b-b6bcf62657d7 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:48:21.305197 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510bc4b9-8bd8-49b8-914b-b6bcf62657d7 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:48:21.305499 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510bc4b9-8bd8-49b8-914b-b6bcf62657d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:48:21.308139 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119804d5-01e6-47f2-b2e9-927fdd83f86f None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:48:21.308422 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119804d5-01e6-47f2-b2e9-927fdd83f86f None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:48:21.308690 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119804d5-01e6-47f2-b2e9-927fdd83f86f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:48:21.327996 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-119804d5-01e6-47f2-b2e9-927fdd83f86f None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:48:21.329198 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-119804d5-01e6-47f2-b2e9-927fdd83f86f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:48:21.335905 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-510bc4b9-8bd8-49b8-914b-b6bcf62657d7 None None] MFA Rules not processed for user `4618d2beb155455fb225613df6a548e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:48:21.337134 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-510bc4b9-8bd8-49b8-914b-b6bcf62657d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:48:21.346024 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:48:21.390149 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1362/2732] 10.222.0.30 () {58 vars in 958 bytes} [Tue Sep 6 04:48:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 04:48:21.399664 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119804d5-01e6-47f2-b2e9-927fdd83f86f None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:48:21.401138 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-119804d5-01e6-47f2-b2e9-927fdd83f86f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:48:21.409245 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:48:21.411544 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1371/2733] 10.222.0.30 () {58 vars in 958 bytes} [Tue Sep 6 04:48:21 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 04:48:21.422402 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510bc4b9-8bd8-49b8-914b-b6bcf62657d7 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:48:21.423819 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-510bc4b9-8bd8-49b8-914b-b6bcf62657d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:48:21.463913 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119804d5-01e6-47f2-b2e9-927fdd83f86f None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:48:21.468399 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7691d50a-7d08-4c75-b51e-10248a7b7fcc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90492440f6a6471e8392361c098428aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b707733ce65c42edbc1fd95ad0a1c06c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:48:21.469279 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7691d50a-7d08-4c75-b51e-10248a7b7fcc service neutron] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:48:21.469589 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7691d50a-7d08-4c75-b51e-10248a7b7fcc service neutron] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:48:21.469864 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7691d50a-7d08-4c75-b51e-10248a7b7fcc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:48:21.478748 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7691d50a-7d08-4c75-b51e-10248a7b7fcc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:48:21.480168 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7691d50a-7d08-4c75-b51e-10248a7b7fcc service neutron] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:48:21.484609 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510bc4b9-8bd8-49b8-914b-b6bcf62657d7 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:48:21.488255 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315be179-41e3-48bd-9231-ac1327c3ba6b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd09b5ab675634a2eb4b03755743f7d0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b707733ce65c42edbc1fd95ad0a1c06c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:48:21.488863 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315be179-41e3-48bd-9231-ac1327c3ba6b service cinder] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:48:21.489143 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315be179-41e3-48bd-9231-ac1327c3ba6b service cinder] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:48:21.489446 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315be179-41e3-48bd-9231-ac1327c3ba6b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:48:21.489913 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1363/2734] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Sep 6 04:48:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 06 04:48:21.554657 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315be179-41e3-48bd-9231-ac1327c3ba6b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:48:21.556115 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315be179-41e3-48bd-9231-ac1327c3ba6b service cinder] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:48:21.566307 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1372/2735] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Sep 6 04:48:21 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 06 04:48:24.604715 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7691d50a-7d08-4c75-b51e-10248a7b7fcc service neutron] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:48:24.616851 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7691d50a-7d08-4c75-b51e-10248a7b7fcc service neutron] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:48:24.622059 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3a2083-f219-4650-ae1d-1cfd4df4eb24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:48:24.622869 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3a2083-f219-4650-ae1d-1cfd4df4eb24 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:48:24.623309 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3a2083-f219-4650-ae1d-1cfd4df4eb24 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:48:24.623610 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3a2083-f219-4650-ae1d-1cfd4df4eb24 admin admin] PATH_INFO: `/v3/users/c8c5e9aad7fa43b6a13404e0859a1f35` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:48:24.625720 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b3a2083-f219-4650-ae1d-1cfd4df4eb24 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c8c5e9aad7fa43b6a13404e0859a1f35)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:48:24.627663 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b3a2083-f219-4650-ae1d-1cfd4df4eb24 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:48:24.794967 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-9b3a2083-f219-4650-ae1d-1cfd4df4eb24 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c8c5e9aad7fa43b6a13404e0859a1f35'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:48:24.865190 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-9b3a2083-f219-4650-ae1d-1cfd4df4eb24 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c8c5e9aad7fa43b6a13404e0859a1f35'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:48:24.865949 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-9b3a2083-f219-4650-ae1d-1cfd4df4eb24 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c8c5e9aad7fa43b6a13404e0859a1f35'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:48:24.874570 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-9b3a2083-f219-4650-ae1d-1cfd4df4eb24 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c8c5e9aad7fa43b6a13404e0859a1f35'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:48:24.889284 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-9b3a2083-f219-4650-ae1d-1cfd4df4eb24 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c8c5e9aad7fa43b6a13404e0859a1f35'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:48:24.896305 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1364/2736] 10.222.0.30 () {58 vars in 1224 bytes} [Tue Sep 6 04:48:24 2022] DELETE /identity/v3/users/c8c5e9aad7fa43b6a13404e0859a1f35 => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 04:48:24.901070 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3eee56b-3030-4a6a-a445-93cbc76e655b None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:48:24.901599 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3eee56b-3030-4a6a-a445-93cbc76e655b None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:48:24.901599 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3eee56b-3030-4a6a-a445-93cbc76e655b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:48:24.926045 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-c3eee56b-3030-4a6a-a445-93cbc76e655b None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:48:24.927416 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-c3eee56b-3030-4a6a-a445-93cbc76e655b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:48:24.989386 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:48:24.999385 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1373/2737] 10.222.0.30 () {58 vars in 958 bytes} [Tue Sep 6 04:48:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 04:48:25.013259 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3a2083-f219-4650-ae1d-1cfd4df4eb24 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:48:25.016225 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-9b3a2083-f219-4650-ae1d-1cfd4df4eb24 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:48:25.082694 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3a2083-f219-4650-ae1d-1cfd4df4eb24 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:48:25.086721 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38cc985-f8aa-4a2e-8a58-a52b71a8c15c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90492440f6a6471e8392361c098428aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b707733ce65c42edbc1fd95ad0a1c06c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:48:25.087466 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38cc985-f8aa-4a2e-8a58-a52b71a8c15c service neutron] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:48:25.087842 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38cc985-f8aa-4a2e-8a58-a52b71a8c15c service neutron] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:48:25.088214 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38cc985-f8aa-4a2e-8a58-a52b71a8c15c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:48:25.098964 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c38cc985-f8aa-4a2e-8a58-a52b71a8c15c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:48:25.100603 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c38cc985-f8aa-4a2e-8a58-a52b71a8c15c service neutron] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:48:25.113187 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1365/2738] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Sep 6 04:48:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 04:48:25.782267 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3eee56b-3030-4a6a-a445-93cbc76e655b None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:48:25.786462 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-c3eee56b-3030-4a6a-a445-93cbc76e655b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:48:25.806650 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3eee56b-3030-4a6a-a445-93cbc76e655b None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:48:25.813406 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffaade83-8b69-4588-ad6e-4ccd452af24f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:48:25.814142 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffaade83-8b69-4588-ad6e-4ccd452af24f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:48:25.814597 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffaade83-8b69-4588-ad6e-4ccd452af24f admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:48:25.815183 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffaade83-8b69-4588-ad6e-4ccd452af24f admin admin] PATH_INFO: `/v3/projects/581650ced13b4553b05dfb09b851da45` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:48:25.816441 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffaade83-8b69-4588-ad6e-4ccd452af24f admin admin] RBAC: Authorizing `identity:delete_project(project_id=581650ced13b4553b05dfb09b851da45)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:48:25.818615 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffaade83-8b69-4588-ad6e-4ccd452af24f admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:48:25.944783 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-ffaade83-8b69-4588-ad6e-4ccd452af24f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '581650ced13b4553b05dfb09b851da45', 'request_id': 'req-ffaade83-8b69-4588-ad6e-4ccd452af24f'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:48:25.951008 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-ffaade83-8b69-4588-ad6e-4ccd452af24f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '581650ced13b4553b05dfb09b851da45', 'request_id': 'req-ffaade83-8b69-4588-ad6e-4ccd452af24f'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:48:25.990735 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-ffaade83-8b69-4588-ad6e-4ccd452af24f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '581650ced13b4553b05dfb09b851da45', 'request_id': 'req-ffaade83-8b69-4588-ad6e-4ccd452af24f'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:48:26.002716 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-ffaade83-8b69-4588-ad6e-4ccd452af24f admin admin] The token cache is being invalidate because project 581650ced13b4553b05dfb09b851da45 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81698) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 04:48:26.003068 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-ffaade83-8b69-4588-ad6e-4ccd452af24f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:48:26.003713 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-ffaade83-8b69-4588-ad6e-4ccd452af24f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:48:26.005571 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1374/2739] 10.222.0.30 () {58 vars in 1233 bytes} [Tue Sep 6 04:48:25 2022] DELETE /identity/v3/projects/581650ced13b4553b05dfb09b851da45 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 04:48:26.013991 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38cc985-f8aa-4a2e-8a58-a52b71a8c15c service neutron] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:48:26.020132 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38cc985-f8aa-4a2e-8a58-a52b71a8c15c service neutron] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:48:26.041796 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:48:26.085240 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46eec45a-51a5-41f8-a718-99a2b14b4e50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:48:26.085928 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46eec45a-51a5-41f8-a718-99a2b14b4e50 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:48:26.086420 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46eec45a-51a5-41f8-a718-99a2b14b4e50 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:48:26.086572 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46eec45a-51a5-41f8-a718-99a2b14b4e50 admin admin] PATH_INFO: `/v3/users/4618d2beb155455fb225613df6a548e2` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:48:26.087992 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46eec45a-51a5-41f8-a718-99a2b14b4e50 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4618d2beb155455fb225613df6a548e2)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:48:26.090163 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46eec45a-51a5-41f8-a718-99a2b14b4e50 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:48:26.300503 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-46eec45a-51a5-41f8-a718-99a2b14b4e50 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4618d2beb155455fb225613df6a548e2'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:48:26.413821 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-46eec45a-51a5-41f8-a718-99a2b14b4e50 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4618d2beb155455fb225613df6a548e2'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:48:26.415142 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-46eec45a-51a5-41f8-a718-99a2b14b4e50 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4618d2beb155455fb225613df6a548e2'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:48:26.431068 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-46eec45a-51a5-41f8-a718-99a2b14b4e50 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4618d2beb155455fb225613df6a548e2'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:48:26.448962 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-46eec45a-51a5-41f8-a718-99a2b14b4e50 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4618d2beb155455fb225613df6a548e2'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:48:26.458489 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1366/2740] 10.222.0.30 () {58 vars in 1224 bytes} [Tue Sep 6 04:48:26 2022] DELETE /identity/v3/users/4618d2beb155455fb225613df6a548e2 => generated 0 bytes in 447 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 04:48:26.525679 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391f1fd2-9774-477f-ae7e-6239a7852c91 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:48:26.525679 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391f1fd2-9774-477f-ae7e-6239a7852c91 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:48:26.525679 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391f1fd2-9774-477f-ae7e-6239a7852c91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:48:26.575002 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-391f1fd2-9774-477f-ae7e-6239a7852c91 None None] MFA Rules not processed for user `31b919a031504f2bace596da457b91cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:48:26.582137 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-391f1fd2-9774-477f-ae7e-6239a7852c91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:48:26.662294 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1375/2741] 10.222.0.30 () {58 vars in 958 bytes} [Tue Sep 6 04:48:26 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 04:48:26.676740 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46eec45a-51a5-41f8-a718-99a2b14b4e50 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:48:26.677157 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-46eec45a-51a5-41f8-a718-99a2b14b4e50 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:48:26.782387 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46eec45a-51a5-41f8-a718-99a2b14b4e50 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:48:26.788065 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36e23d1-a5bc-4371-83d9-3b8723e8006d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9db138b77afa4b068244b8ef715404fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b707733ce65c42edbc1fd95ad0a1c06c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:48:26.788757 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36e23d1-a5bc-4371-83d9-3b8723e8006d service nova] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:48:26.789178 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36e23d1-a5bc-4371-83d9-3b8723e8006d service nova] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:48:26.789628 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36e23d1-a5bc-4371-83d9-3b8723e8006d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:48:26.800476 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d36e23d1-a5bc-4371-83d9-3b8723e8006d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:48:26.802213 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d36e23d1-a5bc-4371-83d9-3b8723e8006d service nova] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:48:26.812501 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1367/2742] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Sep 6 04:48:26 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 04:48:26.925526 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391f1fd2-9774-477f-ae7e-6239a7852c91 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:48:26.927294 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-391f1fd2-9774-477f-ae7e-6239a7852c91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:48:27.009984 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391f1fd2-9774-477f-ae7e-6239a7852c91 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:48:27.015653 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef3f0f2-e831-42e2-bb93-d04a2d76c35f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:48:27.021810 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef3f0f2-e831-42e2-bb93-d04a2d76c35f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:48:27.021810 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef3f0f2-e831-42e2-bb93-d04a2d76c35f admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:48:27.021810 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef3f0f2-e831-42e2-bb93-d04a2d76c35f admin admin] PATH_INFO: `/v3/projects/e253814312db4066839c6c812d05a16c` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:48:27.021810 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef3f0f2-e831-42e2-bb93-d04a2d76c35f admin admin] RBAC: Authorizing `identity:delete_project(project_id=e253814312db4066839c6c812d05a16c)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:48:27.021810 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef3f0f2-e831-42e2-bb93-d04a2d76c35f admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:48:27.038806 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:48:27.141869 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-6ef3f0f2-e831-42e2-bb93-d04a2d76c35f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e253814312db4066839c6c812d05a16c', 'request_id': 'req-6ef3f0f2-e831-42e2-bb93-d04a2d76c35f'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:48:27.147377 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-6ef3f0f2-e831-42e2-bb93-d04a2d76c35f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e253814312db4066839c6c812d05a16c', 'request_id': 'req-6ef3f0f2-e831-42e2-bb93-d04a2d76c35f'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:48:27.219230 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-6ef3f0f2-e831-42e2-bb93-d04a2d76c35f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e253814312db4066839c6c812d05a16c', 'request_id': 'req-6ef3f0f2-e831-42e2-bb93-d04a2d76c35f'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:48:27.232254 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-6ef3f0f2-e831-42e2-bb93-d04a2d76c35f admin admin] The token cache is being invalidate because project e253814312db4066839c6c812d05a16c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81698) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 04:48:27.232552 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-6ef3f0f2-e831-42e2-bb93-d04a2d76c35f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:48:27.233349 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-6ef3f0f2-e831-42e2-bb93-d04a2d76c35f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:48:27.235280 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1376/2743] 10.222.0.30 () {58 vars in 1233 bytes} [Tue Sep 6 04:48:26 2022] DELETE /identity/v3/projects/e253814312db4066839c6c812d05a16c => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 04:48:27.240263 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b374a2ba-c697-44a1-8fd9-e0a739175b0d None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:48:27.240783 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b374a2ba-c697-44a1-8fd9-e0a739175b0d None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:48:27.240891 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b374a2ba-c697-44a1-8fd9-e0a739175b0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:48:27.265745 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-b374a2ba-c697-44a1-8fd9-e0a739175b0d None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:48:27.269556 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-b374a2ba-c697-44a1-8fd9-e0a739175b0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:48:27.290852 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:48:27.333537 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1368/2744] 10.222.0.30 () {58 vars in 958 bytes} [Tue Sep 6 04:48:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 04:48:27.341521 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029b7d06-915f-4a2b-a712-904fc16fa87b None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:48:27.341808 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029b7d06-915f-4a2b-a712-904fc16fa87b None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:48:27.342080 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029b7d06-915f-4a2b-a712-904fc16fa87b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:48:27.364021 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-029b7d06-915f-4a2b-a712-904fc16fa87b None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:48:27.365218 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-029b7d06-915f-4a2b-a712-904fc16fa87b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:48:27.374143 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1377/2745] 10.222.0.30 () {58 vars in 958 bytes} [Tue Sep 6 04:48:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 04:48:27.379883 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b374a2ba-c697-44a1-8fd9-e0a739175b0d None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:48:27.390564 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b374a2ba-c697-44a1-8fd9-e0a739175b0d None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:48:27.395207 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3fba8b-2bfc-4148-982e-2d991274d525 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:48:27.395808 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3fba8b-2bfc-4148-982e-2d991274d525 admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:48:27.396085 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3fba8b-2bfc-4148-982e-2d991274d525 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:48:27.396354 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3fba8b-2bfc-4148-982e-2d991274d525 admin admin] PATH_INFO: `/v3/domains` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:48:27.397026 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed3fba8b-2bfc-4148-982e-2d991274d525 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:48:27.397351 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed3fba8b-2bfc-4148-982e-2d991274d525 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:48:27.398980 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed3fba8b-2bfc-4148-982e-2d991274d525 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:48:27.414658 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1369/2746] 10.222.0.30 () {58 vars in 1153 bytes} [Tue Sep 6 04:48:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:48:27.415964 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029b7d06-915f-4a2b-a712-904fc16fa87b None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:48:27.425393 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029b7d06-915f-4a2b-a712-904fc16fa87b None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:48:27.430119 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c153d2e0-0a41-49c4-98f7-d9163ff9c39c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:48:27.430714 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c153d2e0-0a41-49c4-98f7-d9163ff9c39c admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:48:27.431997 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c153d2e0-0a41-49c4-98f7-d9163ff9c39c admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:48:27.431997 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c153d2e0-0a41-49c4-98f7-d9163ff9c39c admin admin] PATH_INFO: `/v3/projects` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:48:27.431997 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c153d2e0-0a41-49c4-98f7-d9163ff9c39c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:48:27.434063 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c153d2e0-0a41-49c4-98f7-d9163ff9c39c admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:48:27.544482 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1378/2747] 10.222.0.30 () {60 vars in 1153 bytes} [Tue Sep 6 04:48:27 2022] POST /identity/v3/projects => generated 385 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 04:48:27.546503 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3fba8b-2bfc-4148-982e-2d991274d525 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:48:27.559317 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3fba8b-2bfc-4148-982e-2d991274d525 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:48:27.626787 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2aaf9a-87ce-4e1b-b240-9d477856561d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:48:27.627480 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2aaf9a-87ce-4e1b-b240-9d477856561d admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:48:27.627823 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2aaf9a-87ce-4e1b-b240-9d477856561d admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:48:27.628162 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2aaf9a-87ce-4e1b-b240-9d477856561d admin admin] PATH_INFO: `/v3/users` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:48:27.628848 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2aaf9a-87ce-4e1b-b240-9d477856561d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:48:27.631176 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2aaf9a-87ce-4e1b-b240-9d477856561d admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:48:27.728696 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1370/2748] 10.222.0.30 () {60 vars in 1144 bytes} [Tue Sep 6 04:48:27 2022] POST /identity/v3/users => generated 293 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 04:48:27.732718 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c153d2e0-0a41-49c4-98f7-d9163ff9c39c admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:48:27.753967 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c153d2e0-0a41-49c4-98f7-d9163ff9c39c admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:48:27.757325 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e2915e-429e-44c4-8d6b-029528debd84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:48:27.757956 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e2915e-429e-44c4-8d6b-029528debd84 admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:48:27.758384 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e2915e-429e-44c4-8d6b-029528debd84 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:48:27.758777 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e2915e-429e-44c4-8d6b-029528debd84 admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:48:27.759419 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e2915e-429e-44c4-8d6b-029528debd84 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:48:27.761069 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e2915e-429e-44c4-8d6b-029528debd84 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:48:27.774244 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1379/2749] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Sep 6 04:48:27 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 04:48:27.778075 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2aaf9a-87ce-4e1b-b240-9d477856561d admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:48:27.792534 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2aaf9a-87ce-4e1b-b240-9d477856561d admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:48:27.798236 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51f0ad1-eae2-4096-86fa-1e5a2772cac3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:48:27.798884 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51f0ad1-eae2-4096-86fa-1e5a2772cac3 admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:48:27.799313 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51f0ad1-eae2-4096-86fa-1e5a2772cac3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:48:27.800963 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51f0ad1-eae2-4096-86fa-1e5a2772cac3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:48:27.801687 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51f0ad1-eae2-4096-86fa-1e5a2772cac3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:48:27.803330 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51f0ad1-eae2-4096-86fa-1e5a2772cac3 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:48:27.822175 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1371/2750] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Sep 6 04:48:27 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 06 04:48:27.826042 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e2915e-429e-44c4-8d6b-029528debd84 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:48:27.839624 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e2915e-429e-44c4-8d6b-029528debd84 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:48:27.845023 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461f1f9b-07ec-4e18-bfa1-569ef8b5ebe0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:48:27.845739 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461f1f9b-07ec-4e18-bfa1-569ef8b5ebe0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:48:27.846018 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461f1f9b-07ec-4e18-bfa1-569ef8b5ebe0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:48:27.846294 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461f1f9b-07ec-4e18-bfa1-569ef8b5ebe0 admin admin] PATH_INFO: `/v3/projects/c07e9d23d0424a5b9df13d548ad17cde/users/06d18280921440208cb79aeacb77a0d8/roles/29607b2de39f40d68bb6ceef64b8a732` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:48:27.864477 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461f1f9b-07ec-4e18-bfa1-569ef8b5ebe0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c07e9d23d0424a5b9df13d548ad17cde, user_id=06d18280921440208cb79aeacb77a0d8, role_id=29607b2de39f40d68bb6ceef64b8a732)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:48:27.869012 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461f1f9b-07ec-4e18-bfa1-569ef8b5ebe0 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:48:27.955297 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1380/2751] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Sep 6 04:48:27 2022] PUT /identity/v3/projects/c07e9d23d0424a5b9df13d548ad17cde/users/06d18280921440208cb79aeacb77a0d8/roles/29607b2de39f40d68bb6ceef64b8a732 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 04:48:27.965843 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f907109-cde7-455a-b299-4dec53d5fc63 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:48:27.966059 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f907109-cde7-455a-b299-4dec53d5fc63 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:48:27.966331 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f907109-cde7-455a-b299-4dec53d5fc63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:48:28.004488 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-3f907109-cde7-455a-b299-4dec53d5fc63 None None] MFA Rules not processed for user `06d18280921440208cb79aeacb77a0d8`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:48:28.006689 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-3f907109-cde7-455a-b299-4dec53d5fc63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:48:28.084777 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1372/2752] 10.222.0.30 () {58 vars in 958 bytes} [Tue Sep 6 04:48:27 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 04:48:28.089069 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461f1f9b-07ec-4e18-bfa1-569ef8b5ebe0 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:48:28.179649 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461f1f9b-07ec-4e18-bfa1-569ef8b5ebe0 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:48:28.189082 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac95c7c0-5009-4227-9d33-44e180ee5fe2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:48:28.190059 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac95c7c0-5009-4227-9d33-44e180ee5fe2 admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:48:28.190445 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac95c7c0-5009-4227-9d33-44e180ee5fe2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:48:28.190814 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac95c7c0-5009-4227-9d33-44e180ee5fe2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:48:28.191560 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac95c7c0-5009-4227-9d33-44e180ee5fe2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:48:28.193623 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac95c7c0-5009-4227-9d33-44e180ee5fe2 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:48:28.285652 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1381/2753] 10.222.0.30 () {60 vars in 1153 bytes} [Tue Sep 6 04:48:28 2022] POST /identity/v3/projects => generated 385 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 04:48:28.289496 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f907109-cde7-455a-b299-4dec53d5fc63 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:48:28.302447 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f907109-cde7-455a-b299-4dec53d5fc63 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:48:28.325224 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:48:28.376194 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc61bf94-bf08-47ca-bc1b-6b82dfd5d6d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:48:28.377182 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc61bf94-bf08-47ca-bc1b-6b82dfd5d6d6 admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:48:28.378012 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc61bf94-bf08-47ca-bc1b-6b82dfd5d6d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:48:28.378392 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc61bf94-bf08-47ca-bc1b-6b82dfd5d6d6 admin admin] PATH_INFO: `/v3/users` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:48:28.382459 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc61bf94-bf08-47ca-bc1b-6b82dfd5d6d6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:48:28.382459 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc61bf94-bf08-47ca-bc1b-6b82dfd5d6d6 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:48:28.460051 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1373/2754] 10.222.0.30 () {60 vars in 1144 bytes} [Tue Sep 6 04:48:28 2022] POST /identity/v3/users => generated 293 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 04:48:28.464386 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac95c7c0-5009-4227-9d33-44e180ee5fe2 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:48:28.475586 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac95c7c0-5009-4227-9d33-44e180ee5fe2 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:48:28.480513 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3667d9f0-8494-4880-a125-29c9d96f20a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:48:28.481163 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3667d9f0-8494-4880-a125-29c9d96f20a2 admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:48:28.481492 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3667d9f0-8494-4880-a125-29c9d96f20a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:48:28.481768 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3667d9f0-8494-4880-a125-29c9d96f20a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:48:28.482518 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3667d9f0-8494-4880-a125-29c9d96f20a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:48:28.484623 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3667d9f0-8494-4880-a125-29c9d96f20a2 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:48:28.495561 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1382/2755] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Sep 6 04:48:28 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 04:48:28.499764 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc61bf94-bf08-47ca-bc1b-6b82dfd5d6d6 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:48:28.509939 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc61bf94-bf08-47ca-bc1b-6b82dfd5d6d6 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:48:28.515837 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0520b8c5-8c51-4932-a8e9-61efbd9793ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:48:28.516484 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0520b8c5-8c51-4932-a8e9-61efbd9793ed admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:48:28.516762 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0520b8c5-8c51-4932-a8e9-61efbd9793ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:48:28.517029 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0520b8c5-8c51-4932-a8e9-61efbd9793ed admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:48:28.517709 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0520b8c5-8c51-4932-a8e9-61efbd9793ed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:48:28.519309 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0520b8c5-8c51-4932-a8e9-61efbd9793ed admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:48:28.530352 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1374/2756] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Sep 6 04:48:28 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 04:48:28.534896 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3667d9f0-8494-4880-a125-29c9d96f20a2 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:48:28.546963 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3667d9f0-8494-4880-a125-29c9d96f20a2 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:48:28.552296 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf61b5f-e31e-460f-8641-097fa5985c22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:48:28.553107 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf61b5f-e31e-460f-8641-097fa5985c22 admin admin] REQUEST_METHOD: `PUT` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:48:28.553516 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf61b5f-e31e-460f-8641-097fa5985c22 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:48:28.553988 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf61b5f-e31e-460f-8641-097fa5985c22 admin admin] PATH_INFO: `/v3/projects/274ec6db8c064245a37a81f87de92b20/users/02c56cbad48f44d99d5ea675dbf05d12/roles/29607b2de39f40d68bb6ceef64b8a732` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:48:28.568481 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:48:28.573201 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf61b5f-e31e-460f-8641-097fa5985c22 admin admin] RBAC: Authorizing `identity:create_grant(project_id=274ec6db8c064245a37a81f87de92b20, user_id=02c56cbad48f44d99d5ea675dbf05d12, role_id=29607b2de39f40d68bb6ceef64b8a732)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:48:28.576191 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf61b5f-e31e-460f-8641-097fa5985c22 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:48:28.652510 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1383/2757] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Sep 6 04:48:28 2022] PUT /identity/v3/projects/274ec6db8c064245a37a81f87de92b20/users/02c56cbad48f44d99d5ea675dbf05d12/roles/29607b2de39f40d68bb6ceef64b8a732 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 04:48:28.664148 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6129b80b-eb04-48f2-b4ea-9a9114ebeece None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:48:28.664440 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6129b80b-eb04-48f2-b4ea-9a9114ebeece None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:48:28.664718 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6129b80b-eb04-48f2-b4ea-9a9114ebeece None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:48:28.696749 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-6129b80b-eb04-48f2-b4ea-9a9114ebeece None None] MFA Rules not processed for user `02c56cbad48f44d99d5ea675dbf05d12`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:48:28.698030 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-6129b80b-eb04-48f2-b4ea-9a9114ebeece None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:48:28.779556 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1375/2758] 10.222.0.30 () {58 vars in 958 bytes} [Tue Sep 6 04:48:28 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 04:48:28.784354 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf61b5f-e31e-460f-8641-097fa5985c22 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:48:28.863370 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf61b5f-e31e-460f-8641-097fa5985c22 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:48:28.868365 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7760aa1a-678d-4ae2-86c4-8919ff692ea9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:48:28.869057 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7760aa1a-678d-4ae2-86c4-8919ff692ea9 admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:48:28.869440 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7760aa1a-678d-4ae2-86c4-8919ff692ea9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:48:28.869788 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7760aa1a-678d-4ae2-86c4-8919ff692ea9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:48:28.870500 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7760aa1a-678d-4ae2-86c4-8919ff692ea9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:48:28.872435 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7760aa1a-678d-4ae2-86c4-8919ff692ea9 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:48:28.957931 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1384/2759] 10.222.0.30 () {60 vars in 1153 bytes} [Tue Sep 6 04:48:28 2022] POST /identity/v3/projects => generated 383 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 04:48:28.962190 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6129b80b-eb04-48f2-b4ea-9a9114ebeece None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:48:28.979253 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6129b80b-eb04-48f2-b4ea-9a9114ebeece None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:48:29.056487 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07dfbabf-9187-42e6-926b-56038d8d8af8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:48:29.060057 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07dfbabf-9187-42e6-926b-56038d8d8af8 admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:48:29.060057 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07dfbabf-9187-42e6-926b-56038d8d8af8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:48:29.060057 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07dfbabf-9187-42e6-926b-56038d8d8af8 admin admin] PATH_INFO: `/v3/users` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:48:29.060057 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07dfbabf-9187-42e6-926b-56038d8d8af8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:48:29.062717 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07dfbabf-9187-42e6-926b-56038d8d8af8 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:48:29.183530 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1376/2760] 10.222.0.30 () {60 vars in 1144 bytes} [Tue Sep 6 04:48:28 2022] POST /identity/v3/users => generated 298 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 04:48:29.187939 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7760aa1a-678d-4ae2-86c4-8919ff692ea9 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:48:29.198561 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7760aa1a-678d-4ae2-86c4-8919ff692ea9 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:48:29.203904 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce59f08-9c82-4fb5-a778-5cb8b86eaab9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:48:29.204588 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce59f08-9c82-4fb5-a778-5cb8b86eaab9 admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:48:29.204960 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce59f08-9c82-4fb5-a778-5cb8b86eaab9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:48:29.205356 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce59f08-9c82-4fb5-a778-5cb8b86eaab9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:48:29.206084 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce59f08-9c82-4fb5-a778-5cb8b86eaab9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:48:29.207806 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce59f08-9c82-4fb5-a778-5cb8b86eaab9 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:48:29.219463 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1385/2761] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Sep 6 04:48:29 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 06 04:48:29.223731 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07dfbabf-9187-42e6-926b-56038d8d8af8 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:48:29.234129 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07dfbabf-9187-42e6-926b-56038d8d8af8 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:48:29.239184 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a85990f-59cb-431c-957e-714a21ee7f02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:48:29.240042 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a85990f-59cb-431c-957e-714a21ee7f02 admin admin] REQUEST_METHOD: `PUT` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:48:29.240321 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a85990f-59cb-431c-957e-714a21ee7f02 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:48:29.240683 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a85990f-59cb-431c-957e-714a21ee7f02 admin admin] PATH_INFO: `/v3/projects/bb1eeeedbb504eb88d66a5f177b0d028/users/04a86dce52b943839c809d8ca009d061/roles/ea4c814713b149108e488de855b025e4` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:48:29.255569 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a85990f-59cb-431c-957e-714a21ee7f02 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bb1eeeedbb504eb88d66a5f177b0d028, user_id=04a86dce52b943839c809d8ca009d061, role_id=ea4c814713b149108e488de855b025e4)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:48:29.258545 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a85990f-59cb-431c-957e-714a21ee7f02 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:48:29.265919 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:48:29.359364 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1377/2762] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Sep 6 04:48:29 2022] PUT /identity/v3/projects/bb1eeeedbb504eb88d66a5f177b0d028/users/04a86dce52b943839c809d8ca009d061/roles/ea4c814713b149108e488de855b025e4 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 04:48:29.360242 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce59f08-9c82-4fb5-a778-5cb8b86eaab9 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:48:29.371390 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce59f08-9c82-4fb5-a778-5cb8b86eaab9 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:48:29.443692 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db23b742-65a9-4a7d-ba6e-f70c2ebfb22c None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:48:29.443985 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db23b742-65a9-4a7d-ba6e-f70c2ebfb22c None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:48:29.444263 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db23b742-65a9-4a7d-ba6e-f70c2ebfb22c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:48:29.457572 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9dd1dd2-51a3-4d5b-8f15-bd3baf22d083 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:48:29.458204 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9dd1dd2-51a3-4d5b-8f15-bd3baf22d083 admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:48:29.458489 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9dd1dd2-51a3-4d5b-8f15-bd3baf22d083 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:48:29.458767 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9dd1dd2-51a3-4d5b-8f15-bd3baf22d083 admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:48:29.459419 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9dd1dd2-51a3-4d5b-8f15-bd3baf22d083 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:48:29.461036 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9dd1dd2-51a3-4d5b-8f15-bd3baf22d083 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:48:29.473388 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1386/2763] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Sep 6 04:48:29 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 04:48:29.476401 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-db23b742-65a9-4a7d-ba6e-f70c2ebfb22c None None] MFA Rules not processed for user `31b919a031504f2bace596da457b91cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:48:29.477788 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-db23b742-65a9-4a7d-ba6e-f70c2ebfb22c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:48:29.477933 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9dd1dd2-51a3-4d5b-8f15-bd3baf22d083 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:48:29.488591 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9dd1dd2-51a3-4d5b-8f15-bd3baf22d083 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:48:29.493765 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a2cc0d-9464-4fad-88c5-3889cfc41bce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:48:29.494442 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a2cc0d-9464-4fad-88c5-3889cfc41bce admin admin] REQUEST_METHOD: `PUT` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:48:29.494721 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a2cc0d-9464-4fad-88c5-3889cfc41bce admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:48:29.494993 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a2cc0d-9464-4fad-88c5-3889cfc41bce admin admin] PATH_INFO: `/v3/projects/bb1eeeedbb504eb88d66a5f177b0d028/users/04a86dce52b943839c809d8ca009d061/roles/ea4c814713b149108e488de855b025e4` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:48:29.496953 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a2cc0d-9464-4fad-88c5-3889cfc41bce admin admin] RBAC: Authorizing `identity:create_grant(project_id=bb1eeeedbb504eb88d66a5f177b0d028, user_id=04a86dce52b943839c809d8ca009d061, role_id=ea4c814713b149108e488de855b025e4)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:48:29.500239 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a2cc0d-9464-4fad-88c5-3889cfc41bce admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:48:29.522642 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1387/2764] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Sep 6 04:48:29 2022] PUT /identity/v3/projects/bb1eeeedbb504eb88d66a5f177b0d028/users/04a86dce52b943839c809d8ca009d061/roles/ea4c814713b149108e488de855b025e4 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 04:48:29.524478 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e8b3bb-78fa-4e0f-b006-f40d59e14258 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:48:29.524764 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e8b3bb-78fa-4e0f-b006-f40d59e14258 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:48:29.525034 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e8b3bb-78fa-4e0f-b006-f40d59e14258 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:48:29.549855 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1378/2765] 10.222.0.30 () {58 vars in 958 bytes} [Tue Sep 6 04:48:29 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 04:48:29.558012 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-16e8b3bb-78fa-4e0f-b006-f40d59e14258 None None] MFA Rules not processed for user `04a86dce52b943839c809d8ca009d061`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:48:29.559233 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-16e8b3bb-78fa-4e0f-b006-f40d59e14258 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:48:29.606591 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:48:29.652769 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1388/2766] 10.222.0.30 () {58 vars in 958 bytes} [Tue Sep 6 04:48:29 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 04:48:29.662725 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db23b742-65a9-4a7d-ba6e-f70c2ebfb22c None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:48:29.664323 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-db23b742-65a9-4a7d-ba6e-f70c2ebfb22c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:48:29.736372 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db23b742-65a9-4a7d-ba6e-f70c2ebfb22c None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:48:29.816753 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29102184-f521-47a2-aa0b-e0640ee91fca service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd09b5ab675634a2eb4b03755743f7d0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b707733ce65c42edbc1fd95ad0a1c06c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:48:29.817613 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29102184-f521-47a2-aa0b-e0640ee91fca service cinder] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:48:29.822836 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29102184-f521-47a2-aa0b-e0640ee91fca service cinder] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:48:29.822836 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29102184-f521-47a2-aa0b-e0640ee91fca service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:48:29.904794 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29102184-f521-47a2-aa0b-e0640ee91fca service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:48:29.906764 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29102184-f521-47a2-aa0b-e0640ee91fca service cinder] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:48:29.917737 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1379/2767] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Sep 6 04:48:29 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 256 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 06 04:48:29.976931 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e8b3bb-78fa-4e0f-b006-f40d59e14258 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:48:29.979553 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-16e8b3bb-78fa-4e0f-b006-f40d59e14258 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:48:30.063724 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e8b3bb-78fa-4e0f-b006-f40d59e14258 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:48:30.068158 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2b7256-3164-49aa-a832-0aafc3292211 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90492440f6a6471e8392361c098428aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b707733ce65c42edbc1fd95ad0a1c06c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:48:30.069040 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2b7256-3164-49aa-a832-0aafc3292211 service neutron] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:48:30.069520 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2b7256-3164-49aa-a832-0aafc3292211 service neutron] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:48:30.070134 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2b7256-3164-49aa-a832-0aafc3292211 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:48:30.072153 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-7f2b7256-3164-49aa-a832-0aafc3292211 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:48:30.088779 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f2b7256-3164-49aa-a832-0aafc3292211 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:48:30.090531 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f2b7256-3164-49aa-a832-0aafc3292211 service neutron] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:48:30.100707 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1389/2768] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Sep 6 04:48:29 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 04:48:30.621496 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29102184-f521-47a2-aa0b-e0640ee91fca service cinder] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:48:30.621897 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-29102184-f521-47a2-aa0b-e0640ee91fca service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:48:30.635335 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:48:30.707560 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29102184-f521-47a2-aa0b-e0640ee91fca service cinder] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:48:30.713253 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41dff948-be2d-4331-a11e-c97a7195ce39 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d4f2e4dbd41441d84f03ddb30191e7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b707733ce65c42edbc1fd95ad0a1c06c', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:48:30.713925 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41dff948-be2d-4331-a11e-c97a7195ce39 service placement] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:48:30.714191 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41dff948-be2d-4331-a11e-c97a7195ce39 service placement] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:48:30.714444 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41dff948-be2d-4331-a11e-c97a7195ce39 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:48:30.716215 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-41dff948-be2d-4331-a11e-c97a7195ce39 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:48:30.728463 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41dff948-be2d-4331-a11e-c97a7195ce39 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:48:30.731568 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41dff948-be2d-4331-a11e-c97a7195ce39 service placement] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:48:30.740850 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1380/2769] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Sep 6 04:48:30 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 04:48:32.031365 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2b7256-3164-49aa-a832-0aafc3292211 service neutron] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:48:32.032782 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-7f2b7256-3164-49aa-a832-0aafc3292211 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:48:32.044605 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2b7256-3164-49aa-a832-0aafc3292211 service neutron] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:48:32.048789 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3195033-f70f-4e3b-bfd9-fd8b9650f1f8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd09b5ab675634a2eb4b03755743f7d0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b707733ce65c42edbc1fd95ad0a1c06c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:48:32.049421 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3195033-f70f-4e3b-bfd9-fd8b9650f1f8 service cinder] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:48:32.049703 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3195033-f70f-4e3b-bfd9-fd8b9650f1f8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:48:32.049975 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3195033-f70f-4e3b-bfd9-fd8b9650f1f8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:48:32.109637 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3195033-f70f-4e3b-bfd9-fd8b9650f1f8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:48:32.111098 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3195033-f70f-4e3b-bfd9-fd8b9650f1f8 service cinder] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:48:32.120079 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1390/2770] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Sep 6 04:48:32 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 04:48:33.438534 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41dff948-be2d-4331-a11e-c97a7195ce39 service placement] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:48:33.447913 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41dff948-be2d-4331-a11e-c97a7195ce39 service placement] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:48:33.451505 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9fcfd51-8ed3-48ed-abd7-e22846b45442 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd09b5ab675634a2eb4b03755743f7d0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b707733ce65c42edbc1fd95ad0a1c06c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:48:33.452103 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fcfd51-8ed3-48ed-abd7-e22846b45442 service cinder] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:48:33.452426 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fcfd51-8ed3-48ed-abd7-e22846b45442 service cinder] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:48:33.452695 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fcfd51-8ed3-48ed-abd7-e22846b45442 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:48:33.462036 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9fcfd51-8ed3-48ed-abd7-e22846b45442 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:48:33.463523 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9fcfd51-8ed3-48ed-abd7-e22846b45442 service cinder] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:48:33.474243 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1381/2771] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Sep 6 04:48:33 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 04:48:34.026216 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3195033-f70f-4e3b-bfd9-fd8b9650f1f8 service cinder] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:48:34.027972 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-d3195033-f70f-4e3b-bfd9-fd8b9650f1f8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:48:34.046692 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3195033-f70f-4e3b-bfd9-fd8b9650f1f8 service cinder] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:48:34.052013 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f324a3-ec48-48ad-b703-5322d2dd8fce service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90492440f6a6471e8392361c098428aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b707733ce65c42edbc1fd95ad0a1c06c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:48:34.052980 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f324a3-ec48-48ad-b703-5322d2dd8fce service neutron] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:48:34.053265 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f324a3-ec48-48ad-b703-5322d2dd8fce service neutron] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:48:34.053572 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f324a3-ec48-48ad-b703-5322d2dd8fce service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:48:34.055336 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-39f324a3-ec48-48ad-b703-5322d2dd8fce service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:48:34.137955 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:48:34.147822 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f324a3-ec48-48ad-b703-5322d2dd8fce service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:48:34.149266 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f324a3-ec48-48ad-b703-5322d2dd8fce service neutron] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:48:34.169539 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1391/2772] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Sep 6 04:48:34 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 04:48:38.235459 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9fcfd51-8ed3-48ed-abd7-e22846b45442 service cinder] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:48:38.296627 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9fcfd51-8ed3-48ed-abd7-e22846b45442 service cinder] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:48:38.301492 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a7f1cc-b27b-4557-b48a-6de5fd396087 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:48:38.302446 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a7f1cc-b27b-4557-b48a-6de5fd396087 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:48:38.302725 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a7f1cc-b27b-4557-b48a-6de5fd396087 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:48:38.302999 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a7f1cc-b27b-4557-b48a-6de5fd396087 admin admin] PATH_INFO: `/v3/users/06d18280921440208cb79aeacb77a0d8` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:48:38.304104 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a7f1cc-b27b-4557-b48a-6de5fd396087 admin admin] RBAC: Authorizing `identity:delete_user(user_id=06d18280921440208cb79aeacb77a0d8)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:48:38.306066 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a7f1cc-b27b-4557-b48a-6de5fd396087 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:48:38.476335 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-90a7f1cc-b27b-4557-b48a-6de5fd396087 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '06d18280921440208cb79aeacb77a0d8'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:48:38.548423 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-90a7f1cc-b27b-4557-b48a-6de5fd396087 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '06d18280921440208cb79aeacb77a0d8'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:48:38.549036 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-90a7f1cc-b27b-4557-b48a-6de5fd396087 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '06d18280921440208cb79aeacb77a0d8'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:48:38.556844 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-90a7f1cc-b27b-4557-b48a-6de5fd396087 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '06d18280921440208cb79aeacb77a0d8'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:48:38.569450 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-90a7f1cc-b27b-4557-b48a-6de5fd396087 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '06d18280921440208cb79aeacb77a0d8'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:48:38.575251 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1382/2773] 10.222.0.30 () {58 vars in 1224 bytes} [Tue Sep 6 04:48:38 2022] DELETE /identity/v3/users/06d18280921440208cb79aeacb77a0d8 => generated 0 bytes in 340 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 04:48:38.581065 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79d61b1-bb47-4e4b-97ad-cfbebbe59d89 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:48:38.581569 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79d61b1-bb47-4e4b-97ad-cfbebbe59d89 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:48:38.581772 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79d61b1-bb47-4e4b-97ad-cfbebbe59d89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:48:38.600782 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-e79d61b1-bb47-4e4b-97ad-cfbebbe59d89 None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:48:38.602013 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-e79d61b1-bb47-4e4b-97ad-cfbebbe59d89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:48:38.661290 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1392/2774] 10.222.0.30 () {58 vars in 958 bytes} [Tue Sep 6 04:48:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 04:48:38.670963 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a7f1cc-b27b-4557-b48a-6de5fd396087 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:48:38.672365 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-90a7f1cc-b27b-4557-b48a-6de5fd396087 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:48:38.682517 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:48:38.730894 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a7f1cc-b27b-4557-b48a-6de5fd396087 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:48:38.735078 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145c7dfc-5ed7-4355-94cd-2a1f388bcfa7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90492440f6a6471e8392361c098428aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b707733ce65c42edbc1fd95ad0a1c06c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:48:38.735964 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145c7dfc-5ed7-4355-94cd-2a1f388bcfa7 service neutron] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:48:38.736451 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145c7dfc-5ed7-4355-94cd-2a1f388bcfa7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:48:38.736876 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145c7dfc-5ed7-4355-94cd-2a1f388bcfa7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:48:38.746350 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145c7dfc-5ed7-4355-94cd-2a1f388bcfa7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:48:38.747787 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145c7dfc-5ed7-4355-94cd-2a1f388bcfa7 service neutron] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:48:38.757056 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1383/2775] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Sep 6 04:48:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 04:48:39.129635 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79d61b1-bb47-4e4b-97ad-cfbebbe59d89 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:48:39.131046 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-e79d61b1-bb47-4e4b-97ad-cfbebbe59d89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:48:39.145492 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79d61b1-bb47-4e4b-97ad-cfbebbe59d89 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:48:39.150247 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ff85cf-8ee5-45a3-b31f-50a19e481cbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:48:39.150961 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ff85cf-8ee5-45a3-b31f-50a19e481cbf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:48:39.151308 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ff85cf-8ee5-45a3-b31f-50a19e481cbf admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:48:39.151641 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ff85cf-8ee5-45a3-b31f-50a19e481cbf admin admin] PATH_INFO: `/v3/projects/c07e9d23d0424a5b9df13d548ad17cde` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:48:39.153103 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ff85cf-8ee5-45a3-b31f-50a19e481cbf admin admin] RBAC: Authorizing `identity:delete_project(project_id=c07e9d23d0424a5b9df13d548ad17cde)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:48:39.155809 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ff85cf-8ee5-45a3-b31f-50a19e481cbf admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:48:39.249708 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-71ff85cf-8ee5-45a3-b31f-50a19e481cbf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c07e9d23d0424a5b9df13d548ad17cde', 'request_id': 'req-71ff85cf-8ee5-45a3-b31f-50a19e481cbf'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:48:39.257084 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-71ff85cf-8ee5-45a3-b31f-50a19e481cbf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c07e9d23d0424a5b9df13d548ad17cde', 'request_id': 'req-71ff85cf-8ee5-45a3-b31f-50a19e481cbf'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:48:39.299353 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-71ff85cf-8ee5-45a3-b31f-50a19e481cbf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c07e9d23d0424a5b9df13d548ad17cde', 'request_id': 'req-71ff85cf-8ee5-45a3-b31f-50a19e481cbf'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:48:39.312171 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-71ff85cf-8ee5-45a3-b31f-50a19e481cbf admin admin] The token cache is being invalidate because project c07e9d23d0424a5b9df13d548ad17cde was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81698) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 04:48:39.312567 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-71ff85cf-8ee5-45a3-b31f-50a19e481cbf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:48:39.313327 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-71ff85cf-8ee5-45a3-b31f-50a19e481cbf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:48:39.315019 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1393/2776] 10.222.0.30 () {58 vars in 1233 bytes} [Tue Sep 6 04:48:39 2022] DELETE /identity/v3/projects/c07e9d23d0424a5b9df13d548ad17cde => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 04:48:39.319275 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145c7dfc-5ed7-4355-94cd-2a1f388bcfa7 service neutron] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:48:39.329095 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145c7dfc-5ed7-4355-94cd-2a1f388bcfa7 service neutron] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:48:39.400364 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36336271-1afa-4deb-a22a-e9f6e35da5a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:48:39.401012 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36336271-1afa-4deb-a22a-e9f6e35da5a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:48:39.401297 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36336271-1afa-4deb-a22a-e9f6e35da5a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:48:39.401656 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36336271-1afa-4deb-a22a-e9f6e35da5a6 admin admin] PATH_INFO: `/v3/users/02c56cbad48f44d99d5ea675dbf05d12` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:48:39.405479 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36336271-1afa-4deb-a22a-e9f6e35da5a6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=02c56cbad48f44d99d5ea675dbf05d12)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:48:39.405479 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36336271-1afa-4deb-a22a-e9f6e35da5a6 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:48:39.619507 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-36336271-1afa-4deb-a22a-e9f6e35da5a6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '02c56cbad48f44d99d5ea675dbf05d12'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:48:39.734145 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-36336271-1afa-4deb-a22a-e9f6e35da5a6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '02c56cbad48f44d99d5ea675dbf05d12'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:48:39.738236 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-36336271-1afa-4deb-a22a-e9f6e35da5a6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '02c56cbad48f44d99d5ea675dbf05d12'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:48:39.744931 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-36336271-1afa-4deb-a22a-e9f6e35da5a6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '02c56cbad48f44d99d5ea675dbf05d12'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:48:39.766538 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-36336271-1afa-4deb-a22a-e9f6e35da5a6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '02c56cbad48f44d99d5ea675dbf05d12'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:48:39.779287 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1384/2777] 10.222.0.30 () {58 vars in 1224 bytes} [Tue Sep 6 04:48:39 2022] DELETE /identity/v3/users/02c56cbad48f44d99d5ea675dbf05d12 => generated 0 bytes in 458 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 04:48:39.969437 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ff85cf-8ee5-45a3-b31f-50a19e481cbf admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:48:39.976171 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-71ff85cf-8ee5-45a3-b31f-50a19e481cbf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:48:40.071948 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:48:40.077262 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ff85cf-8ee5-45a3-b31f-50a19e481cbf admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:48:40.176535 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ac39eb-87df-4c44-aac0-719e7836675c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9db138b77afa4b068244b8ef715404fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b707733ce65c42edbc1fd95ad0a1c06c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:48:40.177563 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ac39eb-87df-4c44-aac0-719e7836675c service nova] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:48:40.177937 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ac39eb-87df-4c44-aac0-719e7836675c service nova] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:48:40.178282 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ac39eb-87df-4c44-aac0-719e7836675c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:48:40.180300 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-f0ac39eb-87df-4c44-aac0-719e7836675c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:48:40.198308 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ac39eb-87df-4c44-aac0-719e7836675c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:48:40.202060 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ac39eb-87df-4c44-aac0-719e7836675c service nova] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:48:40.216267 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1394/2778] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Sep 6 04:48:39 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 248 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 04:48:40.520533 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36336271-1afa-4deb-a22a-e9f6e35da5a6 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:48:40.520533 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-36336271-1afa-4deb-a22a-e9f6e35da5a6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:48:40.560723 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:48:40.599268 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36336271-1afa-4deb-a22a-e9f6e35da5a6 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:48:40.607146 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc1b26f-3a23-417e-88b8-312780e2435c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:48:40.607741 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc1b26f-3a23-417e-88b8-312780e2435c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:48:40.608023 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc1b26f-3a23-417e-88b8-312780e2435c admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:48:40.608298 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc1b26f-3a23-417e-88b8-312780e2435c admin admin] PATH_INFO: `/v3/projects/274ec6db8c064245a37a81f87de92b20` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:48:40.609641 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc1b26f-3a23-417e-88b8-312780e2435c admin admin] RBAC: Authorizing `identity:delete_project(project_id=274ec6db8c064245a37a81f87de92b20)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:48:40.611543 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc1b26f-3a23-417e-88b8-312780e2435c admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:48:40.764303 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-1bc1b26f-3a23-417e-88b8-312780e2435c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '274ec6db8c064245a37a81f87de92b20', 'request_id': 'req-1bc1b26f-3a23-417e-88b8-312780e2435c'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:48:40.879569 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-1bc1b26f-3a23-417e-88b8-312780e2435c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '274ec6db8c064245a37a81f87de92b20', 'request_id': 'req-1bc1b26f-3a23-417e-88b8-312780e2435c'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:48:40.886426 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-1bc1b26f-3a23-417e-88b8-312780e2435c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '274ec6db8c064245a37a81f87de92b20', 'request_id': 'req-1bc1b26f-3a23-417e-88b8-312780e2435c'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:48:40.896948 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-1bc1b26f-3a23-417e-88b8-312780e2435c admin admin] The token cache is being invalidate because project 274ec6db8c064245a37a81f87de92b20 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81697) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 04:48:40.897240 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-1bc1b26f-3a23-417e-88b8-312780e2435c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:48:40.897911 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-1bc1b26f-3a23-417e-88b8-312780e2435c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:48:40.899384 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1385/2779] 10.222.0.30 () {58 vars in 1233 bytes} [Tue Sep 6 04:48:40 2022] DELETE /identity/v3/projects/274ec6db8c064245a37a81f87de92b20 => generated 0 bytes in 388 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 04:48:40.903695 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ac39eb-87df-4c44-aac0-719e7836675c service nova] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:48:40.915750 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ac39eb-87df-4c44-aac0-719e7836675c service nova] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:48:40.992126 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471bd147-d457-4972-b049-bf9ceaaa490d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:48:40.992822 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471bd147-d457-4972-b049-bf9ceaaa490d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:48:40.993098 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471bd147-d457-4972-b049-bf9ceaaa490d admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:48:40.993409 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471bd147-d457-4972-b049-bf9ceaaa490d admin admin] PATH_INFO: `/v3/users/04a86dce52b943839c809d8ca009d061` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:48:40.994558 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471bd147-d457-4972-b049-bf9ceaaa490d admin admin] RBAC: Authorizing `identity:delete_user(user_id=04a86dce52b943839c809d8ca009d061)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:48:40.996767 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471bd147-d457-4972-b049-bf9ceaaa490d admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:48:41.187887 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-471bd147-d457-4972-b049-bf9ceaaa490d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '04a86dce52b943839c809d8ca009d061'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:48:41.270812 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-471bd147-d457-4972-b049-bf9ceaaa490d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '04a86dce52b943839c809d8ca009d061'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:48:41.271428 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-471bd147-d457-4972-b049-bf9ceaaa490d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '04a86dce52b943839c809d8ca009d061'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:48:41.286271 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-471bd147-d457-4972-b049-bf9ceaaa490d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '04a86dce52b943839c809d8ca009d061'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:48:41.293122 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-471bd147-d457-4972-b049-bf9ceaaa490d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '04a86dce52b943839c809d8ca009d061'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:48:41.299112 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1395/2780] 10.222.0.30 () {58 vars in 1224 bytes} [Tue Sep 6 04:48:40 2022] DELETE /identity/v3/users/04a86dce52b943839c809d8ca009d061 => generated 0 bytes in 396 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 04:48:42.083742 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc1b26f-3a23-417e-88b8-312780e2435c admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:48:42.085239 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-1bc1b26f-3a23-417e-88b8-312780e2435c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:48:42.098976 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc1b26f-3a23-417e-88b8-312780e2435c admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:48:42.174391 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6b345a-fc26-4e96-b357-f6b4d558fd70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:48:42.175159 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6b345a-fc26-4e96-b357-f6b4d558fd70 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:48:42.175627 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6b345a-fc26-4e96-b357-f6b4d558fd70 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:48:42.175903 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6b345a-fc26-4e96-b357-f6b4d558fd70 admin admin] PATH_INFO: `/v3/projects/bb1eeeedbb504eb88d66a5f177b0d028` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:48:42.177139 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6b345a-fc26-4e96-b357-f6b4d558fd70 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bb1eeeedbb504eb88d66a5f177b0d028)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:48:42.190345 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6b345a-fc26-4e96-b357-f6b4d558fd70 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:48:42.289262 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-8c6b345a-fc26-4e96-b357-f6b4d558fd70 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bb1eeeedbb504eb88d66a5f177b0d028', 'request_id': 'req-8c6b345a-fc26-4e96-b357-f6b4d558fd70'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:48:42.354192 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-8c6b345a-fc26-4e96-b357-f6b4d558fd70 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bb1eeeedbb504eb88d66a5f177b0d028', 'request_id': 'req-8c6b345a-fc26-4e96-b357-f6b4d558fd70'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:48:42.361755 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-8c6b345a-fc26-4e96-b357-f6b4d558fd70 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bb1eeeedbb504eb88d66a5f177b0d028', 'request_id': 'req-8c6b345a-fc26-4e96-b357-f6b4d558fd70'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:48:42.363069 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:48:42.372919 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-8c6b345a-fc26-4e96-b357-f6b4d558fd70 admin admin] The token cache is being invalidate because project bb1eeeedbb504eb88d66a5f177b0d028 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81697) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 04:48:42.375072 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-8c6b345a-fc26-4e96-b357-f6b4d558fd70 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:48:42.375072 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-8c6b345a-fc26-4e96-b357-f6b4d558fd70 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:48:42.375590 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1386/2781] 10.222.0.30 () {58 vars in 1233 bytes} [Tue Sep 6 04:48:42 2022] DELETE /identity/v3/projects/bb1eeeedbb504eb88d66a5f177b0d028 => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 04:48:51.295544 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=85,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:48:52.572956 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:DELETE=2,keystone:SELECT=3 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:48:52.699218 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471bd147-d457-4972-b049-bf9ceaaa490d admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:48:52.700657 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-471bd147-d457-4972-b049-bf9ceaaa490d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:48:52.758782 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471bd147-d457-4972-b049-bf9ceaaa490d admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:48:52.806806 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59970519-8b39-4882-bc88-3ddd53f4ea6e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90492440f6a6471e8392361c098428aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b707733ce65c42edbc1fd95ad0a1c06c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:48:52.807574 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59970519-8b39-4882-bc88-3ddd53f4ea6e service neutron] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:48:52.807854 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59970519-8b39-4882-bc88-3ddd53f4ea6e service neutron] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:48:52.808124 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59970519-8b39-4882-bc88-3ddd53f4ea6e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:48:52.809886 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-59970519-8b39-4882-bc88-3ddd53f4ea6e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:48:52.872027 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59970519-8b39-4882-bc88-3ddd53f4ea6e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:48:52.873472 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59970519-8b39-4882-bc88-3ddd53f4ea6e service neutron] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:48:52.883652 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1396/2782] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Sep 6 04:48:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 04:48:59.949204 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6b345a-fc26-4e96-b357-f6b4d558fd70 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:48:59.951076 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-8c6b345a-fc26-4e96-b357-f6b4d558fd70 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:48:59.965253 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6b345a-fc26-4e96-b357-f6b4d558fd70 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:48:59.969927 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c4449c-9ae1-4812-8a8a-1056c3716a95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:48:59.970595 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c4449c-9ae1-4812-8a8a-1056c3716a95 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:48:59.970879 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c4449c-9ae1-4812-8a8a-1056c3716a95 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:48:59.971154 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c4449c-9ae1-4812-8a8a-1056c3716a95 admin admin] PATH_INFO: `/v3/users/4b6c761af9c5457ea1e481ce400aded8` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:48:59.972335 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c4449c-9ae1-4812-8a8a-1056c3716a95 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4b6c761af9c5457ea1e481ce400aded8)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:48:59.975501 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c4449c-9ae1-4812-8a8a-1056c3716a95 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:49:00.190626 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-d8c4449c-9ae1-4812-8a8a-1056c3716a95 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4b6c761af9c5457ea1e481ce400aded8'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:49:00.250363 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-d8c4449c-9ae1-4812-8a8a-1056c3716a95 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4b6c761af9c5457ea1e481ce400aded8'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:49:00.250979 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-d8c4449c-9ae1-4812-8a8a-1056c3716a95 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4b6c761af9c5457ea1e481ce400aded8'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:49:00.258329 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-d8c4449c-9ae1-4812-8a8a-1056c3716a95 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4b6c761af9c5457ea1e481ce400aded8'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:49:00.270400 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-d8c4449c-9ae1-4812-8a8a-1056c3716a95 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4b6c761af9c5457ea1e481ce400aded8'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:49:00.276326 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1387/2783] 10.222.0.30 () {58 vars in 1224 bytes} [Tue Sep 6 04:48:59 2022] DELETE /identity/v3/users/4b6c761af9c5457ea1e481ce400aded8 => generated 0 bytes in 328 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 04:49:00.517590 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59970519-8b39-4882-bc88-3ddd53f4ea6e service neutron] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:49:00.535916 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59970519-8b39-4882-bc88-3ddd53f4ea6e service neutron] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:49:00.585839 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95179b46-5bd6-45ef-995d-3ad971305383 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:49:00.586466 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95179b46-5bd6-45ef-995d-3ad971305383 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:49:00.586741 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95179b46-5bd6-45ef-995d-3ad971305383 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:49:00.587009 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95179b46-5bd6-45ef-995d-3ad971305383 admin admin] PATH_INFO: `/v3/projects/375efd2890fc4b71af13439813f00ab0` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:49:00.588144 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95179b46-5bd6-45ef-995d-3ad971305383 admin admin] RBAC: Authorizing `identity:delete_project(project_id=375efd2890fc4b71af13439813f00ab0)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:49:00.590185 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95179b46-5bd6-45ef-995d-3ad971305383 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:49:00.597872 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=100 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:49:00.721683 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-95179b46-5bd6-45ef-995d-3ad971305383 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '375efd2890fc4b71af13439813f00ab0', 'request_id': 'req-95179b46-5bd6-45ef-995d-3ad971305383'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:49:00.727762 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-95179b46-5bd6-45ef-995d-3ad971305383 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '375efd2890fc4b71af13439813f00ab0', 'request_id': 'req-95179b46-5bd6-45ef-995d-3ad971305383'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:49:00.795557 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-95179b46-5bd6-45ef-995d-3ad971305383 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '375efd2890fc4b71af13439813f00ab0', 'request_id': 'req-95179b46-5bd6-45ef-995d-3ad971305383'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:49:00.807815 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-95179b46-5bd6-45ef-995d-3ad971305383 admin admin] The token cache is being invalidate because project 375efd2890fc4b71af13439813f00ab0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81698) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 04:49:00.810926 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-95179b46-5bd6-45ef-995d-3ad971305383 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:49:00.810926 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-95179b46-5bd6-45ef-995d-3ad971305383 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:49:00.810926 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1397/2784] 10.222.0.30 () {58 vars in 1233 bytes} [Tue Sep 6 04:49:00 2022] DELETE /identity/v3/projects/375efd2890fc4b71af13439813f00ab0 => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 04:49:00.814396 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c4449c-9ae1-4812-8a8a-1056c3716a95 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:49:00.815852 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-d8c4449c-9ae1-4812-8a8a-1056c3716a95 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:49:00.829359 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c4449c-9ae1-4812-8a8a-1056c3716a95 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:49:00.885973 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93453204-6ab9-4116-8824-e8910f47b174 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:49:00.886601 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93453204-6ab9-4116-8824-e8910f47b174 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:49:00.886875 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93453204-6ab9-4116-8824-e8910f47b174 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:49:00.887144 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93453204-6ab9-4116-8824-e8910f47b174 admin admin] PATH_INFO: `/v3/users/d8f4eb17e3a843c7abb537e13744be55` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:49:00.899819 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93453204-6ab9-4116-8824-e8910f47b174 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d8f4eb17e3a843c7abb537e13744be55)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:49:00.901751 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93453204-6ab9-4116-8824-e8910f47b174 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:49:01.071835 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-93453204-6ab9-4116-8824-e8910f47b174 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd8f4eb17e3a843c7abb537e13744be55'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:49:01.138508 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-93453204-6ab9-4116-8824-e8910f47b174 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd8f4eb17e3a843c7abb537e13744be55'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:49:01.139129 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-93453204-6ab9-4116-8824-e8910f47b174 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd8f4eb17e3a843c7abb537e13744be55'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:49:01.146639 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-93453204-6ab9-4116-8824-e8910f47b174 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd8f4eb17e3a843c7abb537e13744be55'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:49:01.147739 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=80,keystone:DELETE=18,keystone:INSERT=2 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:49:01.158923 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-93453204-6ab9-4116-8824-e8910f47b174 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd8f4eb17e3a843c7abb537e13744be55'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:49:01.165022 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1388/2785] 10.222.0.30 () {58 vars in 1224 bytes} [Tue Sep 6 04:49:00 2022] DELETE /identity/v3/users/d8f4eb17e3a843c7abb537e13744be55 => generated 0 bytes in 351 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 04:49:01.425749 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95179b46-5bd6-45ef-995d-3ad971305383 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:49:01.427789 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-95179b46-5bd6-45ef-995d-3ad971305383 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:49:01.441346 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95179b46-5bd6-45ef-995d-3ad971305383 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:49:01.499881 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae984c52-0b54-4195-afaf-fcc7e72c99e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:49:01.500552 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae984c52-0b54-4195-afaf-fcc7e72c99e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:49:01.500833 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae984c52-0b54-4195-afaf-fcc7e72c99e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:49:01.501107 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae984c52-0b54-4195-afaf-fcc7e72c99e1 admin admin] PATH_INFO: `/v3/projects/7191b1fbea424d14b85ea63448f414be` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:49:01.511580 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae984c52-0b54-4195-afaf-fcc7e72c99e1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7191b1fbea424d14b85ea63448f414be)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:49:01.513578 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae984c52-0b54-4195-afaf-fcc7e72c99e1 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:49:01.624619 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-ae984c52-0b54-4195-afaf-fcc7e72c99e1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7191b1fbea424d14b85ea63448f414be', 'request_id': 'req-ae984c52-0b54-4195-afaf-fcc7e72c99e1'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:49:01.630682 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-ae984c52-0b54-4195-afaf-fcc7e72c99e1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7191b1fbea424d14b85ea63448f414be', 'request_id': 'req-ae984c52-0b54-4195-afaf-fcc7e72c99e1'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:49:01.703941 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-ae984c52-0b54-4195-afaf-fcc7e72c99e1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7191b1fbea424d14b85ea63448f414be', 'request_id': 'req-ae984c52-0b54-4195-afaf-fcc7e72c99e1'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:49:01.717019 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-ae984c52-0b54-4195-afaf-fcc7e72c99e1 admin admin] The token cache is being invalidate because project 7191b1fbea424d14b85ea63448f414be was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81698) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 04:49:01.717321 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-ae984c52-0b54-4195-afaf-fcc7e72c99e1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:49:01.717967 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-ae984c52-0b54-4195-afaf-fcc7e72c99e1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:49:01.719427 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1398/2786] 10.222.0.30 () {58 vars in 1233 bytes} [Tue Sep 6 04:49:01 2022] DELETE /identity/v3/projects/7191b1fbea424d14b85ea63448f414be => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 04:49:11.252771 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=7,keystone:DELETE=2 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:49:11.712878 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=76,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:49:39.077550 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93453204-6ab9-4116-8824-e8910f47b174 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:49:39.079025 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-93453204-6ab9-4116-8824-e8910f47b174 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:49:39.148126 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93453204-6ab9-4116-8824-e8910f47b174 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:49:39.206285 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736a4402-25d6-45ac-9ccc-36f8cc912d71 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d4f2e4dbd41441d84f03ddb30191e7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b707733ce65c42edbc1fd95ad0a1c06c', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:49:39.206892 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736a4402-25d6-45ac-9ccc-36f8cc912d71 service placement] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:49:39.207174 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736a4402-25d6-45ac-9ccc-36f8cc912d71 service placement] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:49:39.207450 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736a4402-25d6-45ac-9ccc-36f8cc912d71 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:49:39.209171 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-736a4402-25d6-45ac-9ccc-36f8cc912d71 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:49:39.220659 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-736a4402-25d6-45ac-9ccc-36f8cc912d71 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:49:39.222262 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-736a4402-25d6-45ac-9ccc-36f8cc912d71 service placement] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:49:39.232548 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1389/2787] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Sep 6 04:49:39 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 04:49:49.227771 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=58 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:51:21.777480 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae984c52-0b54-4195-afaf-fcc7e72c99e1 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:51:21.778870 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-ae984c52-0b54-4195-afaf-fcc7e72c99e1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:51:21.838006 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae984c52-0b54-4195-afaf-fcc7e72c99e1 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:51:21.843482 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b286b5d-e194-4dfd-ab71-d9cbae9ff08f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd09b5ab675634a2eb4b03755743f7d0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b707733ce65c42edbc1fd95ad0a1c06c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:51:21.844086 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b286b5d-e194-4dfd-ab71-d9cbae9ff08f service cinder] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:51:21.844357 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b286b5d-e194-4dfd-ab71-d9cbae9ff08f service cinder] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:51:21.844663 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b286b5d-e194-4dfd-ab71-d9cbae9ff08f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:51:21.846502 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-4b286b5d-e194-4dfd-ab71-d9cbae9ff08f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:51:21.916004 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b286b5d-e194-4dfd-ab71-d9cbae9ff08f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:51:21.917474 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b286b5d-e194-4dfd-ab71-d9cbae9ff08f service cinder] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:51:21.928242 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1399/2788] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Sep 6 04:51:21 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 04:51:25.855338 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736a4402-25d6-45ac-9ccc-36f8cc912d71 service placement] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:51:25.856732 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-736a4402-25d6-45ac-9ccc-36f8cc912d71 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:51:25.912214 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736a4402-25d6-45ac-9ccc-36f8cc912d71 service placement] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:51:25.915760 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15a69af-89e1-4041-96fd-36aa881e23f5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90492440f6a6471e8392361c098428aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b707733ce65c42edbc1fd95ad0a1c06c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:51:25.916351 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15a69af-89e1-4041-96fd-36aa881e23f5 service neutron] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:51:25.916623 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15a69af-89e1-4041-96fd-36aa881e23f5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:51:25.916888 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15a69af-89e1-4041-96fd-36aa881e23f5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:51:25.918592 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-d15a69af-89e1-4041-96fd-36aa881e23f5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:51:25.932186 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15a69af-89e1-4041-96fd-36aa881e23f5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:51:25.933619 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15a69af-89e1-4041-96fd-36aa881e23f5 service neutron] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:51:25.945023 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1390/2789] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Sep 6 04:51:25 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 04:51:31.923395 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=54 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:51:35.940205 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=26 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:51:36.089859 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b286b5d-e194-4dfd-ab71-d9cbae9ff08f service cinder] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:51:36.099380 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b286b5d-e194-4dfd-ab71-d9cbae9ff08f service cinder] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:51:36.102957 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6dd7d3-7d75-4b42-8a06-14c82dac12d0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90492440f6a6471e8392361c098428aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b707733ce65c42edbc1fd95ad0a1c06c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:51:36.103551 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6dd7d3-7d75-4b42-8a06-14c82dac12d0 service neutron] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:51:36.103828 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6dd7d3-7d75-4b42-8a06-14c82dac12d0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:51:36.104100 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6dd7d3-7d75-4b42-8a06-14c82dac12d0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:51:36.105854 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-5b6dd7d3-7d75-4b42-8a06-14c82dac12d0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:51:36.115260 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6dd7d3-7d75-4b42-8a06-14c82dac12d0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:51:36.116663 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6dd7d3-7d75-4b42-8a06-14c82dac12d0 service neutron] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:51:36.124521 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1400/2790] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Sep 6 04:51:36 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 04:51:46.120976 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=6 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:52:45.114890 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15a69af-89e1-4041-96fd-36aa881e23f5 service neutron] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:52:45.116373 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-d15a69af-89e1-4041-96fd-36aa881e23f5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:52:45.130748 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15a69af-89e1-4041-96fd-36aa881e23f5 service neutron] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:52:45.136375 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61da77e3-7c1c-454e-8355-68fecc41ad4b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9db138b77afa4b068244b8ef715404fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b707733ce65c42edbc1fd95ad0a1c06c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:52:45.136982 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61da77e3-7c1c-454e-8355-68fecc41ad4b service nova] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:52:45.137263 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61da77e3-7c1c-454e-8355-68fecc41ad4b service nova] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:52:45.137563 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61da77e3-7c1c-454e-8355-68fecc41ad4b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:52:45.139275 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-61da77e3-7c1c-454e-8355-68fecc41ad4b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:52:45.151764 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61da77e3-7c1c-454e-8355-68fecc41ad4b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:52:45.153425 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61da77e3-7c1c-454e-8355-68fecc41ad4b service nova] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:52:45.166320 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1391/2791] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Sep 6 04:52:45 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 06 04:52:49.745214 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad53efc2-b92f-4606-b0fc-21e47e965d25 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:52:49.746134 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad53efc2-b92f-4606-b0fc-21e47e965d25 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:52:49.746134 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad53efc2-b92f-4606-b0fc-21e47e965d25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:52:49.767523 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-ad53efc2-b92f-4606-b0fc-21e47e965d25 None None] MFA Rules not processed for user `31b919a031504f2bace596da457b91cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:52:49.768417 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-ad53efc2-b92f-4606-b0fc-21e47e965d25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:52:49.828523 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1401/2792] 10.222.0.30 () {58 vars in 958 bytes} [Tue Sep 6 04:52:49 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 04:52:55.161225 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=6 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:52:59.821756 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=25 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:54:21.084264 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61da77e3-7c1c-454e-8355-68fecc41ad4b service nova] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:54:21.087913 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-61da77e3-7c1c-454e-8355-68fecc41ad4b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:54:21.178628 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61da77e3-7c1c-454e-8355-68fecc41ad4b service nova] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:54:21.184450 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd84457-2ca0-403f-af96-5fc7cc0f9bcc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9db138b77afa4b068244b8ef715404fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b707733ce65c42edbc1fd95ad0a1c06c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:54:21.185062 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd84457-2ca0-403f-af96-5fc7cc0f9bcc service nova] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:54:21.185376 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd84457-2ca0-403f-af96-5fc7cc0f9bcc service nova] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:54:21.185661 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd84457-2ca0-403f-af96-5fc7cc0f9bcc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:54:21.187452 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-8fd84457-2ca0-403f-af96-5fc7cc0f9bcc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:54:21.198129 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd84457-2ca0-403f-af96-5fc7cc0f9bcc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:54:21.199726 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd84457-2ca0-403f-af96-5fc7cc0f9bcc service nova] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:54:21.208848 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1392/2793] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Sep 6 04:54:21 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 04:54:31.204770 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=40 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:54:41.410801 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad53efc2-b92f-4606-b0fc-21e47e965d25 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:54:41.412261 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-ad53efc2-b92f-4606-b0fc-21e47e965d25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:54:41.476379 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad53efc2-b92f-4606-b0fc-21e47e965d25 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:54:41.482140 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e224c8b-f3db-44bd-953c-4832b9a93580 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d4f2e4dbd41441d84f03ddb30191e7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b707733ce65c42edbc1fd95ad0a1c06c', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:54:41.482740 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e224c8b-f3db-44bd-953c-4832b9a93580 service placement] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:54:41.483017 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e224c8b-f3db-44bd-953c-4832b9a93580 service placement] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:54:41.483285 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e224c8b-f3db-44bd-953c-4832b9a93580 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:54:41.492137 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e224c8b-f3db-44bd-953c-4832b9a93580 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:54:41.493860 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e224c8b-f3db-44bd-953c-4832b9a93580 service placement] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:54:41.503675 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1402/2794] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Sep 6 04:54:41 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 04:54:51.499528 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=27 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:54:57.876700 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd84457-2ca0-403f-af96-5fc7cc0f9bcc service nova] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:54:57.895431 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd84457-2ca0-403f-af96-5fc7cc0f9bcc service nova] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:54:57.899315 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7054a9f2-a4a4-4be9-8f6b-387a3c226aef service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90492440f6a6471e8392361c098428aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b707733ce65c42edbc1fd95ad0a1c06c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:54:57.899939 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7054a9f2-a4a4-4be9-8f6b-387a3c226aef service neutron] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:54:57.900532 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7054a9f2-a4a4-4be9-8f6b-387a3c226aef service neutron] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:54:57.900747 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7054a9f2-a4a4-4be9-8f6b-387a3c226aef service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:54:57.902525 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-7054a9f2-a4a4-4be9-8f6b-387a3c226aef service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:54:57.988082 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7054a9f2-a4a4-4be9-8f6b-387a3c226aef service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:54:57.989551 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7054a9f2-a4a4-4be9-8f6b-387a3c226aef service neutron] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:54:58.008911 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1393/2795] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Sep 6 04:54:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 04:55:01.326035 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e224c8b-f3db-44bd-953c-4832b9a93580 service placement] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:55:01.339005 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e224c8b-f3db-44bd-953c-4832b9a93580 service placement] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:55:01.344230 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ed509e-db86-49c2-8310-66fc81558923 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d4f2e4dbd41441d84f03ddb30191e7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b707733ce65c42edbc1fd95ad0a1c06c', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:55:01.344874 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ed509e-db86-49c2-8310-66fc81558923 service placement] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:55:01.345150 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ed509e-db86-49c2-8310-66fc81558923 service placement] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:55:01.345524 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ed509e-db86-49c2-8310-66fc81558923 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:55:01.347240 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-d2ed509e-db86-49c2-8310-66fc81558923 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:55:01.361722 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ed509e-db86-49c2-8310-66fc81558923 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:55:01.363484 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ed509e-db86-49c2-8310-66fc81558923 service placement] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:55:01.378621 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1403/2796] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Sep 6 04:55:01 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 04:55:01.723273 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7054a9f2-a4a4-4be9-8f6b-387a3c226aef service neutron] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:55:01.727324 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-7054a9f2-a4a4-4be9-8f6b-387a3c226aef service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:55:01.738254 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7054a9f2-a4a4-4be9-8f6b-387a3c226aef service neutron] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:55:01.743005 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b5adab-4593-4d16-8dfc-d63ab0097e9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:55:01.744213 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b5adab-4593-4d16-8dfc-d63ab0097e9b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:55:01.744598 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b5adab-4593-4d16-8dfc-d63ab0097e9b admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:55:01.744879 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b5adab-4593-4d16-8dfc-d63ab0097e9b admin admin] PATH_INFO: `/v3/users/31b919a031504f2bace596da457b91cf` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:55:01.746052 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b5adab-4593-4d16-8dfc-d63ab0097e9b admin admin] RBAC: Authorizing `identity:delete_user(user_id=31b919a031504f2bace596da457b91cf)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:55:01.747956 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b5adab-4593-4d16-8dfc-d63ab0097e9b admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:55:01.966410 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-46b5adab-4593-4d16-8dfc-d63ab0097e9b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '31b919a031504f2bace596da457b91cf'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:55:02.037060 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-46b5adab-4593-4d16-8dfc-d63ab0097e9b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '31b919a031504f2bace596da457b91cf'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:55:02.037730 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-46b5adab-4593-4d16-8dfc-d63ab0097e9b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '31b919a031504f2bace596da457b91cf'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:55:02.045942 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-46b5adab-4593-4d16-8dfc-d63ab0097e9b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '31b919a031504f2bace596da457b91cf'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:55:02.057740 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-46b5adab-4593-4d16-8dfc-d63ab0097e9b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '31b919a031504f2bace596da457b91cf'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:55:02.063531 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1394/2797] 10.222.0.30 () {58 vars in 1224 bytes} [Tue Sep 6 04:55:01 2022] DELETE /identity/v3/users/31b919a031504f2bace596da457b91cf => generated 0 bytes in 341 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 04:55:02.311222 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ed509e-db86-49c2-8310-66fc81558923 service placement] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:55:02.324564 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ed509e-db86-49c2-8310-66fc81558923 service placement] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:55:02.381897 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03c7291-8c56-416a-8395-1aee7932f55b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:55:02.382565 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03c7291-8c56-416a-8395-1aee7932f55b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:55:02.382806 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03c7291-8c56-416a-8395-1aee7932f55b admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:55:02.383096 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03c7291-8c56-416a-8395-1aee7932f55b admin admin] PATH_INFO: `/v3/projects/f91efa94deff4ceb80b0a1aa88087421` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:55:02.384791 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03c7291-8c56-416a-8395-1aee7932f55b admin admin] RBAC: Authorizing `identity:delete_project(project_id=f91efa94deff4ceb80b0a1aa88087421)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:55:02.388332 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03c7291-8c56-416a-8395-1aee7932f55b admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:55:02.488237 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-d03c7291-8c56-416a-8395-1aee7932f55b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f91efa94deff4ceb80b0a1aa88087421', 'request_id': 'req-d03c7291-8c56-416a-8395-1aee7932f55b'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:55:02.493552 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-d03c7291-8c56-416a-8395-1aee7932f55b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f91efa94deff4ceb80b0a1aa88087421', 'request_id': 'req-d03c7291-8c56-416a-8395-1aee7932f55b'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:55:02.562041 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-d03c7291-8c56-416a-8395-1aee7932f55b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f91efa94deff4ceb80b0a1aa88087421', 'request_id': 'req-d03c7291-8c56-416a-8395-1aee7932f55b'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:55:02.574230 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-d03c7291-8c56-416a-8395-1aee7932f55b admin admin] The token cache is being invalidate because project f91efa94deff4ceb80b0a1aa88087421 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81698) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 04:55:02.574516 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-d03c7291-8c56-416a-8395-1aee7932f55b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:55:02.575130 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-d03c7291-8c56-416a-8395-1aee7932f55b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:55:02.576607 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1404/2798] 10.222.0.30 () {58 vars in 1233 bytes} [Tue Sep 6 04:55:02 2022] DELETE /identity/v3/projects/f91efa94deff4ceb80b0a1aa88087421 => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 04:55:02.581536 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39894cee-a63c-4f44-becb-e86146dd4636 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:55:02.581823 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39894cee-a63c-4f44-becb-e86146dd4636 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:55:02.583079 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39894cee-a63c-4f44-becb-e86146dd4636 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:55:02.621480 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:55:02.641486 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-39894cee-a63c-4f44-becb-e86146dd4636 None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:55:02.642690 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-39894cee-a63c-4f44-becb-e86146dd4636 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:55:02.701150 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1395/2799] 10.222.0.30 () {58 vars in 958 bytes} [Tue Sep 6 04:55:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 04:55:02.709256 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205ab73f-8ddf-4c5f-9e47-eba761669b4e None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:55:02.709573 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205ab73f-8ddf-4c5f-9e47-eba761669b4e None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:55:02.709845 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205ab73f-8ddf-4c5f-9e47-eba761669b4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:55:02.729554 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-205ab73f-8ddf-4c5f-9e47-eba761669b4e None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:55:02.730739 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-205ab73f-8ddf-4c5f-9e47-eba761669b4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:55:02.738620 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1405/2800] 10.222.0.30 () {58 vars in 958 bytes} [Tue Sep 6 04:55:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 04:55:02.742715 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39894cee-a63c-4f44-becb-e86146dd4636 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:55:02.752768 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39894cee-a63c-4f44-becb-e86146dd4636 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:55:02.757181 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895066f1-c6b5-4d3d-9a34-825370f49ac0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:55:02.757824 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895066f1-c6b5-4d3d-9a34-825370f49ac0 admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:55:02.758101 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895066f1-c6b5-4d3d-9a34-825370f49ac0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:55:02.758365 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895066f1-c6b5-4d3d-9a34-825370f49ac0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:55:02.759004 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895066f1-c6b5-4d3d-9a34-825370f49ac0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:55:02.759973 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895066f1-c6b5-4d3d-9a34-825370f49ac0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:55:02.761679 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895066f1-c6b5-4d3d-9a34-825370f49ac0 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:55:02.773276 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1396/2801] 10.222.0.30 () {58 vars in 1153 bytes} [Tue Sep 6 04:55:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:55:02.778255 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205ab73f-8ddf-4c5f-9e47-eba761669b4e None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:55:02.787557 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205ab73f-8ddf-4c5f-9e47-eba761669b4e None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:55:02.792837 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a887480-94fb-4753-8880-cab43b56387c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:55:02.793471 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a887480-94fb-4753-8880-cab43b56387c admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:55:02.793750 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a887480-94fb-4753-8880-cab43b56387c admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:55:02.794020 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a887480-94fb-4753-8880-cab43b56387c admin admin] PATH_INFO: `/v3/projects` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:55:02.794648 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a887480-94fb-4753-8880-cab43b56387c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:55:02.796536 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a887480-94fb-4753-8880-cab43b56387c admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:55:02.877218 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1406/2802] 10.222.0.30 () {60 vars in 1153 bytes} [Tue Sep 6 04:55:02 2022] POST /identity/v3/projects => generated 375 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 04:55:02.881294 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895066f1-c6b5-4d3d-9a34-825370f49ac0 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:55:02.891684 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895066f1-c6b5-4d3d-9a34-825370f49ac0 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:55:02.946191 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79473fc2-ab7a-43e7-9bcd-5ad81d3bfac6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:55:02.947464 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79473fc2-ab7a-43e7-9bcd-5ad81d3bfac6 admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:55:02.947759 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79473fc2-ab7a-43e7-9bcd-5ad81d3bfac6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:55:02.948026 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79473fc2-ab7a-43e7-9bcd-5ad81d3bfac6 admin admin] PATH_INFO: `/v3/users` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:55:02.948641 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79473fc2-ab7a-43e7-9bcd-5ad81d3bfac6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:55:02.950567 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79473fc2-ab7a-43e7-9bcd-5ad81d3bfac6 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:55:03.058626 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1397/2803] 10.222.0.30 () {60 vars in 1144 bytes} [Tue Sep 6 04:55:02 2022] POST /identity/v3/users => generated 288 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 04:55:03.062822 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a887480-94fb-4753-8880-cab43b56387c admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:55:03.073652 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a887480-94fb-4753-8880-cab43b56387c admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:55:03.078284 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a607e577-3dca-4185-ad17-d5e0d0384d6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:55:03.078885 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a607e577-3dca-4185-ad17-d5e0d0384d6a admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:55:03.079165 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a607e577-3dca-4185-ad17-d5e0d0384d6a admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:55:03.079433 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a607e577-3dca-4185-ad17-d5e0d0384d6a admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:55:03.080074 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a607e577-3dca-4185-ad17-d5e0d0384d6a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:55:03.081714 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a607e577-3dca-4185-ad17-d5e0d0384d6a admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:55:03.092478 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1407/2804] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Sep 6 04:55:03 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 04:55:03.096493 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79473fc2-ab7a-43e7-9bcd-5ad81d3bfac6 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:55:03.106872 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79473fc2-ab7a-43e7-9bcd-5ad81d3bfac6 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:55:03.111328 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd88817-ecfc-4352-81f8-c88859d300f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:55:03.111919 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd88817-ecfc-4352-81f8-c88859d300f6 admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:55:03.112197 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd88817-ecfc-4352-81f8-c88859d300f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:55:03.112467 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd88817-ecfc-4352-81f8-c88859d300f6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:55:03.113100 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd88817-ecfc-4352-81f8-c88859d300f6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:55:03.114723 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd88817-ecfc-4352-81f8-c88859d300f6 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:55:03.125637 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1398/2805] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Sep 6 04:55:03 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 04:55:03.129521 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a607e577-3dca-4185-ad17-d5e0d0384d6a admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:55:03.139615 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a607e577-3dca-4185-ad17-d5e0d0384d6a admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:55:03.144176 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87bdc9d-1ce2-423a-b924-f31ead4458bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:55:03.144818 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87bdc9d-1ce2-423a-b924-f31ead4458bd admin admin] REQUEST_METHOD: `PUT` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:55:03.145100 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87bdc9d-1ce2-423a-b924-f31ead4458bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:55:03.145410 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87bdc9d-1ce2-423a-b924-f31ead4458bd admin admin] PATH_INFO: `/v3/projects/c53397010e774adb8adb866bc2c886e7/users/07bac56b8ebf436189819d6cb0f0d8dc/roles/29607b2de39f40d68bb6ceef64b8a732` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:55:03.159943 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a87bdc9d-1ce2-423a-b924-f31ead4458bd admin admin] RBAC: Authorizing `identity:create_grant(project_id=c53397010e774adb8adb866bc2c886e7, user_id=07bac56b8ebf436189819d6cb0f0d8dc, role_id=29607b2de39f40d68bb6ceef64b8a732)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:55:03.162630 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a87bdc9d-1ce2-423a-b924-f31ead4458bd admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:55:03.253024 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1408/2806] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Sep 6 04:55:03 2022] PUT /identity/v3/projects/c53397010e774adb8adb866bc2c886e7/users/07bac56b8ebf436189819d6cb0f0d8dc/roles/29607b2de39f40d68bb6ceef64b8a732 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 04:55:03.264874 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cc32b3-66dd-4be9-bb5e-8183636cfbad None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:55:03.265163 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cc32b3-66dd-4be9-bb5e-8183636cfbad None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:55:03.265477 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cc32b3-66dd-4be9-bb5e-8183636cfbad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:55:03.293291 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-81cc32b3-66dd-4be9-bb5e-8183636cfbad None None] MFA Rules not processed for user `07bac56b8ebf436189819d6cb0f0d8dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:55:03.294514 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-81cc32b3-66dd-4be9-bb5e-8183636cfbad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:55:03.348486 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:55:03.359185 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1399/2807] 10.222.0.30 () {58 vars in 958 bytes} [Tue Sep 6 04:55:03 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 04:55:03.370820 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87bdc9d-1ce2-423a-b924-f31ead4458bd admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:55:03.372535 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-a87bdc9d-1ce2-423a-b924-f31ead4458bd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:55:03.388162 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:55:03.447533 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87bdc9d-1ce2-423a-b924-f31ead4458bd admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:55:03.453084 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3ac05d-8a2b-4165-b020-e70d1cee0cf6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9db138b77afa4b068244b8ef715404fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b707733ce65c42edbc1fd95ad0a1c06c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:55:03.453726 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3ac05d-8a2b-4165-b020-e70d1cee0cf6 service nova] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:55:03.454008 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3ac05d-8a2b-4165-b020-e70d1cee0cf6 service nova] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:55:03.454282 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3ac05d-8a2b-4165-b020-e70d1cee0cf6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:55:03.461853 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3ac05d-8a2b-4165-b020-e70d1cee0cf6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:55:03.464013 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3ac05d-8a2b-4165-b020-e70d1cee0cf6 service nova] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:55:03.473607 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1409/2808] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Sep 6 04:55:03 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 04:55:03.682398 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81cc32b3-66dd-4be9-bb5e-8183636cfbad None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:55:03.743598 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81cc32b3-66dd-4be9-bb5e-8183636cfbad None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:55:03.748056 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41383fe2-6e2f-4333-9dce-58f008f13cef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:55:03.748682 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41383fe2-6e2f-4333-9dce-58f008f13cef admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:55:03.748959 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41383fe2-6e2f-4333-9dce-58f008f13cef admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:55:03.749226 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41383fe2-6e2f-4333-9dce-58f008f13cef admin admin] PATH_INFO: `/v3/users/07bac56b8ebf436189819d6cb0f0d8dc` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:55:03.750393 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41383fe2-6e2f-4333-9dce-58f008f13cef admin admin] RBAC: Authorizing `identity:delete_user(user_id=07bac56b8ebf436189819d6cb0f0d8dc)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:55:03.752285 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41383fe2-6e2f-4333-9dce-58f008f13cef admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:55:03.976060 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-41383fe2-6e2f-4333-9dce-58f008f13cef admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '07bac56b8ebf436189819d6cb0f0d8dc'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:55:04.053625 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-41383fe2-6e2f-4333-9dce-58f008f13cef admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '07bac56b8ebf436189819d6cb0f0d8dc'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:55:04.054268 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-41383fe2-6e2f-4333-9dce-58f008f13cef admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '07bac56b8ebf436189819d6cb0f0d8dc'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:55:04.061679 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-41383fe2-6e2f-4333-9dce-58f008f13cef admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '07bac56b8ebf436189819d6cb0f0d8dc'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:55:04.074128 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-41383fe2-6e2f-4333-9dce-58f008f13cef admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '07bac56b8ebf436189819d6cb0f0d8dc'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:55:04.080200 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1400/2809] 10.222.0.30 () {58 vars in 1224 bytes} [Tue Sep 6 04:55:03 2022] DELETE /identity/v3/users/07bac56b8ebf436189819d6cb0f0d8dc => generated 0 bytes in 399 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 04:55:04.084706 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fb1c0e-0ddc-4da0-9410-c68187f02bbe None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:55:04.085001 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fb1c0e-0ddc-4da0-9410-c68187f02bbe None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:55:04.085273 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fb1c0e-0ddc-4da0-9410-c68187f02bbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:55:04.105389 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-e4fb1c0e-0ddc-4da0-9410-c68187f02bbe None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:55:04.106590 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-e4fb1c0e-0ddc-4da0-9410-c68187f02bbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:55:04.165124 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1410/2810] 10.222.0.30 () {58 vars in 958 bytes} [Tue Sep 6 04:55:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 04:55:04.174818 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41383fe2-6e2f-4333-9dce-58f008f13cef admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:55:04.176136 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-41383fe2-6e2f-4333-9dce-58f008f13cef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:55:04.244825 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41383fe2-6e2f-4333-9dce-58f008f13cef admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:55:04.248447 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f88906-b27c-45ba-be3a-56d31d8a61ab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90492440f6a6471e8392361c098428aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b707733ce65c42edbc1fd95ad0a1c06c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:55:04.249073 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f88906-b27c-45ba-be3a-56d31d8a61ab service neutron] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:55:04.249383 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f88906-b27c-45ba-be3a-56d31d8a61ab service neutron] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:55:04.249654 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f88906-b27c-45ba-be3a-56d31d8a61ab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:55:04.258438 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f88906-b27c-45ba-be3a-56d31d8a61ab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:55:04.260754 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f88906-b27c-45ba-be3a-56d31d8a61ab service neutron] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:55:04.270042 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1401/2811] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Sep 6 04:55:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 04:55:04.621801 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4fb1c0e-0ddc-4da0-9410-c68187f02bbe None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:55:04.629025 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-e4fb1c0e-0ddc-4da0-9410-c68187f02bbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:55:04.635801 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4fb1c0e-0ddc-4da0-9410-c68187f02bbe None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:55:04.640446 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e74e863-33c0-4642-9888-2d7168ae6dff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:55:04.641115 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e74e863-33c0-4642-9888-2d7168ae6dff admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:55:04.641657 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e74e863-33c0-4642-9888-2d7168ae6dff admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:55:04.642114 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e74e863-33c0-4642-9888-2d7168ae6dff admin admin] PATH_INFO: `/v3/projects/c53397010e774adb8adb866bc2c886e7` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:55:04.643577 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e74e863-33c0-4642-9888-2d7168ae6dff admin admin] RBAC: Authorizing `identity:delete_project(project_id=c53397010e774adb8adb866bc2c886e7)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:55:04.645522 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e74e863-33c0-4642-9888-2d7168ae6dff admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:55:04.768734 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-0e74e863-33c0-4642-9888-2d7168ae6dff admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c53397010e774adb8adb866bc2c886e7', 'request_id': 'req-0e74e863-33c0-4642-9888-2d7168ae6dff'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:55:04.775546 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-0e74e863-33c0-4642-9888-2d7168ae6dff admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c53397010e774adb8adb866bc2c886e7', 'request_id': 'req-0e74e863-33c0-4642-9888-2d7168ae6dff'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:55:04.848016 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-0e74e863-33c0-4642-9888-2d7168ae6dff admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c53397010e774adb8adb866bc2c886e7', 'request_id': 'req-0e74e863-33c0-4642-9888-2d7168ae6dff'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:55:04.860259 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-0e74e863-33c0-4642-9888-2d7168ae6dff admin admin] The token cache is being invalidate because project c53397010e774adb8adb866bc2c886e7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81698) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 04:55:04.860927 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-0e74e863-33c0-4642-9888-2d7168ae6dff admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:55:04.861172 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-0e74e863-33c0-4642-9888-2d7168ae6dff admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:55:04.862689 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1411/2812] 10.222.0.30 () {58 vars in 1233 bytes} [Tue Sep 6 04:55:04 2022] DELETE /identity/v3/projects/c53397010e774adb8adb866bc2c886e7 => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 04:55:04.867504 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aea6478-6c57-42a7-8e34-2974da46f444 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:55:04.867917 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aea6478-6c57-42a7-8e34-2974da46f444 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:55:04.868072 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aea6478-6c57-42a7-8e34-2974da46f444 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:55:04.877690 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:55:04.888593 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-2aea6478-6c57-42a7-8e34-2974da46f444 None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:55:04.889826 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-2aea6478-6c57-42a7-8e34-2974da46f444 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:55:04.948753 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1402/2813] 10.222.0.30 () {58 vars in 958 bytes} [Tue Sep 6 04:55:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 04:55:04.956697 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b67929-0863-46d0-96f1-6e72033ab640 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:55:04.956988 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b67929-0863-46d0-96f1-6e72033ab640 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:55:04.957266 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b67929-0863-46d0-96f1-6e72033ab640 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:55:04.977505 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-25b67929-0863-46d0-96f1-6e72033ab640 None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:55:04.979344 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-25b67929-0863-46d0-96f1-6e72033ab640 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:55:04.987222 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1412/2814] 10.222.0.30 () {58 vars in 958 bytes} [Tue Sep 6 04:55:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 04:55:04.991423 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aea6478-6c57-42a7-8e34-2974da46f444 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:55:05.001646 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aea6478-6c57-42a7-8e34-2974da46f444 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:55:05.006200 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df416c0-0dba-40ad-9585-08a06180d6c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:55:05.006800 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df416c0-0dba-40ad-9585-08a06180d6c8 admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:55:05.007080 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df416c0-0dba-40ad-9585-08a06180d6c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:55:05.007351 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df416c0-0dba-40ad-9585-08a06180d6c8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:55:05.007987 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df416c0-0dba-40ad-9585-08a06180d6c8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:55:05.008289 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df416c0-0dba-40ad-9585-08a06180d6c8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:55:05.010227 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df416c0-0dba-40ad-9585-08a06180d6c8 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:55:05.025403 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1403/2815] 10.222.0.30 () {58 vars in 1153 bytes} [Tue Sep 6 04:55:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:55:05.026907 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b67929-0863-46d0-96f1-6e72033ab640 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:55:05.036527 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b67929-0863-46d0-96f1-6e72033ab640 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:55:05.041341 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d54321-dd90-4ce7-a3cb-a6c25dba25be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:55:05.041936 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d54321-dd90-4ce7-a3cb-a6c25dba25be admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:55:05.042253 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d54321-dd90-4ce7-a3cb-a6c25dba25be admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:55:05.042524 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d54321-dd90-4ce7-a3cb-a6c25dba25be admin admin] PATH_INFO: `/v3/projects` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:55:05.043144 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d54321-dd90-4ce7-a3cb-a6c25dba25be admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:55:05.045023 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d54321-dd90-4ce7-a3cb-a6c25dba25be admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:55:05.060133 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:55:05.130372 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1413/2816] 10.222.0.30 () {60 vars in 1153 bytes} [Tue Sep 6 04:55:05 2022] POST /identity/v3/projects => generated 405 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 04:55:05.135505 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df416c0-0dba-40ad-9585-08a06180d6c8 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:55:05.147961 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df416c0-0dba-40ad-9585-08a06180d6c8 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:55:05.207559 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a0d3ff-feb9-49b9-8bf8-fde101aa7054 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:55:05.208164 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a0d3ff-feb9-49b9-8bf8-fde101aa7054 admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:55:05.208439 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a0d3ff-feb9-49b9-8bf8-fde101aa7054 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:55:05.208706 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a0d3ff-feb9-49b9-8bf8-fde101aa7054 admin admin] PATH_INFO: `/v3/users` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:55:05.209360 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a0d3ff-feb9-49b9-8bf8-fde101aa7054 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:55:05.211249 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a0d3ff-feb9-49b9-8bf8-fde101aa7054 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:55:05.321299 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1404/2817] 10.222.0.30 () {60 vars in 1144 bytes} [Tue Sep 6 04:55:05 2022] POST /identity/v3/users => generated 303 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 04:55:05.325745 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d54321-dd90-4ce7-a3cb-a6c25dba25be admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:55:05.336080 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d54321-dd90-4ce7-a3cb-a6c25dba25be admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:55:05.340961 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff682ff0-7ac5-44ed-8361-31f7a0758354 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:55:05.341598 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff682ff0-7ac5-44ed-8361-31f7a0758354 admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:55:05.341879 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff682ff0-7ac5-44ed-8361-31f7a0758354 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:55:05.342150 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff682ff0-7ac5-44ed-8361-31f7a0758354 admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:55:05.342791 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff682ff0-7ac5-44ed-8361-31f7a0758354 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:55:05.344454 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff682ff0-7ac5-44ed-8361-31f7a0758354 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:55:05.355072 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1414/2818] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Sep 6 04:55:05 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 06 04:55:05.359119 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a0d3ff-feb9-49b9-8bf8-fde101aa7054 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:55:05.372444 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a0d3ff-feb9-49b9-8bf8-fde101aa7054 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:55:05.377646 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c7ca06-03db-439a-913f-c02d374cdb2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:55:05.379301 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c7ca06-03db-439a-913f-c02d374cdb2d admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:55:05.379301 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c7ca06-03db-439a-913f-c02d374cdb2d admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:55:05.379301 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c7ca06-03db-439a-913f-c02d374cdb2d admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:55:05.379654 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c7ca06-03db-439a-913f-c02d374cdb2d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:55:05.381085 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c7ca06-03db-439a-913f-c02d374cdb2d admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:55:05.393883 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1405/2819] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Sep 6 04:55:05 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 04:55:05.398051 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff682ff0-7ac5-44ed-8361-31f7a0758354 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:55:05.407993 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff682ff0-7ac5-44ed-8361-31f7a0758354 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:55:05.412841 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c7b09a-2e1c-4345-a568-6b7a3a2b54c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:55:05.413504 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c7b09a-2e1c-4345-a568-6b7a3a2b54c7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:55:05.413781 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c7b09a-2e1c-4345-a568-6b7a3a2b54c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:55:05.414048 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c7b09a-2e1c-4345-a568-6b7a3a2b54c7 admin admin] PATH_INFO: `/v3/projects/2e29862416e748bdb5322ee789510ea5/users/412b8a91889249b196cfe6daac1bcf20/roles/29607b2de39f40d68bb6ceef64b8a732` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:55:05.430867 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c7b09a-2e1c-4345-a568-6b7a3a2b54c7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2e29862416e748bdb5322ee789510ea5, user_id=412b8a91889249b196cfe6daac1bcf20, role_id=29607b2de39f40d68bb6ceef64b8a732)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:55:05.434227 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c7b09a-2e1c-4345-a568-6b7a3a2b54c7 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:55:05.503628 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1415/2820] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Sep 6 04:55:05 2022] PUT /identity/v3/projects/2e29862416e748bdb5322ee789510ea5/users/412b8a91889249b196cfe6daac1bcf20/roles/29607b2de39f40d68bb6ceef64b8a732 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 04:55:05.508670 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7d6c68-a8e8-4061-900f-a5d3846dde83 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:55:05.508964 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7d6c68-a8e8-4061-900f-a5d3846dde83 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:55:05.509278 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7d6c68-a8e8-4061-900f-a5d3846dde83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:55:05.527929 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-dc7d6c68-a8e8-4061-900f-a5d3846dde83 None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:55:05.529176 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-dc7d6c68-a8e8-4061-900f-a5d3846dde83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:55:05.587863 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1406/2821] 10.222.0.30 () {58 vars in 958 bytes} [Tue Sep 6 04:55:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 04:55:05.597288 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c7b09a-2e1c-4345-a568-6b7a3a2b54c7 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:55:05.599001 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-f4c7b09a-2e1c-4345-a568-6b7a3a2b54c7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:55:05.658063 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c7b09a-2e1c-4345-a568-6b7a3a2b54c7 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:55:05.661964 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03435987-cf37-47b9-acc0-162a44113f92 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90492440f6a6471e8392361c098428aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b707733ce65c42edbc1fd95ad0a1c06c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:55:05.662568 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03435987-cf37-47b9-acc0-162a44113f92 service neutron] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:55:05.662848 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03435987-cf37-47b9-acc0-162a44113f92 service neutron] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:55:05.663116 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03435987-cf37-47b9-acc0-162a44113f92 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:55:05.672872 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03435987-cf37-47b9-acc0-162a44113f92 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:55:05.674303 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03435987-cf37-47b9-acc0-162a44113f92 service neutron] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:55:05.683935 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1416/2822] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Sep 6 04:55:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 04:55:06.871586 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff934c7-da8d-42ab-9736-10704552f88b None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:55:06.872059 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff934c7-da8d-42ab-9736-10704552f88b None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:55:06.872421 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff934c7-da8d-42ab-9736-10704552f88b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:55:06.893485 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:55:06.901857 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-2ff934c7-da8d-42ab-9736-10704552f88b None None] MFA Rules not processed for user `412b8a91889249b196cfe6daac1bcf20`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:55:06.903088 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-2ff934c7-da8d-42ab-9736-10704552f88b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:55:06.969067 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1407/2823] 10.222.0.30 () {58 vars in 958 bytes} [Tue Sep 6 04:55:06 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 04:55:06.973895 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03435987-cf37-47b9-acc0-162a44113f92 service neutron] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:55:06.984003 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03435987-cf37-47b9-acc0-162a44113f92 service neutron] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:55:06.988836 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9ffcef-bb59-4aae-9cb7-ac362783eb90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:55:06.989536 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9ffcef-bb59-4aae-9cb7-ac362783eb90 admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:55:06.989813 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9ffcef-bb59-4aae-9cb7-ac362783eb90 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:55:06.990080 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9ffcef-bb59-4aae-9cb7-ac362783eb90 admin admin] PATH_INFO: `/v3/projects` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:55:06.990711 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9ffcef-bb59-4aae-9cb7-ac362783eb90 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:55:06.992624 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9ffcef-bb59-4aae-9cb7-ac362783eb90 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:55:07.094739 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1417/2824] 10.222.0.30 () {60 vars in 1153 bytes} [Tue Sep 6 04:55:06 2022] POST /identity/v3/projects => generated 405 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 04:55:07.099011 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff934c7-da8d-42ab-9736-10704552f88b None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:55:07.110604 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff934c7-da8d-42ab-9736-10704552f88b None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:55:07.167243 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7baf35de-8c33-44ec-b20f-ddf5246d36ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:55:07.167791 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7baf35de-8c33-44ec-b20f-ddf5246d36ec admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:55:07.168069 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7baf35de-8c33-44ec-b20f-ddf5246d36ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:55:07.168338 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7baf35de-8c33-44ec-b20f-ddf5246d36ec admin admin] PATH_INFO: `/v3/users` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:55:07.168976 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7baf35de-8c33-44ec-b20f-ddf5246d36ec admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:55:07.171010 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7baf35de-8c33-44ec-b20f-ddf5246d36ec admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:55:07.247885 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1408/2825] 10.222.0.30 () {60 vars in 1144 bytes} [Tue Sep 6 04:55:07 2022] POST /identity/v3/users => generated 309 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 04:55:07.251940 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9ffcef-bb59-4aae-9cb7-ac362783eb90 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:55:07.262737 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9ffcef-bb59-4aae-9cb7-ac362783eb90 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:55:07.267280 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b6eca5-7b18-4d16-92c5-d451d7e4b56a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:55:07.267872 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b6eca5-7b18-4d16-92c5-d451d7e4b56a admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:55:07.268148 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b6eca5-7b18-4d16-92c5-d451d7e4b56a admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:55:07.268452 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b6eca5-7b18-4d16-92c5-d451d7e4b56a admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:55:07.269080 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b6eca5-7b18-4d16-92c5-d451d7e4b56a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:55:07.270716 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b6eca5-7b18-4d16-92c5-d451d7e4b56a admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:55:07.281181 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1418/2826] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Sep 6 04:55:07 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 04:55:07.285478 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7baf35de-8c33-44ec-b20f-ddf5246d36ec admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:55:07.295472 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7baf35de-8c33-44ec-b20f-ddf5246d36ec admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:55:07.300083 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf7a76a-33ee-4826-9f5b-1b2d7ea3428b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:55:07.300725 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf7a76a-33ee-4826-9f5b-1b2d7ea3428b admin admin] REQUEST_METHOD: `PUT` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:55:07.301003 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf7a76a-33ee-4826-9f5b-1b2d7ea3428b admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:55:07.301273 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf7a76a-33ee-4826-9f5b-1b2d7ea3428b admin admin] PATH_INFO: `/v3/projects/4ca92f8b7d6c40839abf33f31d7d55f5/users/0030134573474da397006175a79243a8/roles/ea4c814713b149108e488de855b025e4` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:55:07.316688 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf7a76a-33ee-4826-9f5b-1b2d7ea3428b admin admin] RBAC: Authorizing `identity:create_grant(project_id=4ca92f8b7d6c40839abf33f31d7d55f5, user_id=0030134573474da397006175a79243a8, role_id=ea4c814713b149108e488de855b025e4)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:55:07.319519 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf7a76a-33ee-4826-9f5b-1b2d7ea3428b admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:55:07.402751 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1409/2827] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Sep 6 04:55:07 2022] PUT /identity/v3/projects/4ca92f8b7d6c40839abf33f31d7d55f5/users/0030134573474da397006175a79243a8/roles/ea4c814713b149108e488de855b025e4 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 04:55:07.407887 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b6eca5-7b18-4d16-92c5-d451d7e4b56a admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:55:07.416767 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b6eca5-7b18-4d16-92c5-d451d7e4b56a admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:55:07.471160 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb29b24-c7e2-4dab-98e0-2d96ab1334ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:55:07.471698 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb29b24-c7e2-4dab-98e0-2d96ab1334ff admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:55:07.471977 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb29b24-c7e2-4dab-98e0-2d96ab1334ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:55:07.472246 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb29b24-c7e2-4dab-98e0-2d96ab1334ff admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:55:07.472888 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb29b24-c7e2-4dab-98e0-2d96ab1334ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:55:07.474602 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb29b24-c7e2-4dab-98e0-2d96ab1334ff admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:55:07.484448 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1419/2828] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Sep 6 04:55:07 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 04:55:07.488679 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf7a76a-33ee-4826-9f5b-1b2d7ea3428b admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:55:07.499516 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf7a76a-33ee-4826-9f5b-1b2d7ea3428b admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:55:07.504050 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe6f3ea-137d-4b26-a28e-5b8d38f51748 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:55:07.504721 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe6f3ea-137d-4b26-a28e-5b8d38f51748 admin admin] REQUEST_METHOD: `PUT` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:55:07.504997 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe6f3ea-137d-4b26-a28e-5b8d38f51748 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:55:07.505263 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe6f3ea-137d-4b26-a28e-5b8d38f51748 admin admin] PATH_INFO: `/v3/projects/4ca92f8b7d6c40839abf33f31d7d55f5/users/0030134573474da397006175a79243a8/roles/ea4c814713b149108e488de855b025e4` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:55:07.507421 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe6f3ea-137d-4b26-a28e-5b8d38f51748 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4ca92f8b7d6c40839abf33f31d7d55f5, user_id=0030134573474da397006175a79243a8, role_id=ea4c814713b149108e488de855b025e4)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:55:07.510093 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe6f3ea-137d-4b26-a28e-5b8d38f51748 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:55:07.520641 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1410/2829] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Sep 6 04:55:07 2022] PUT /identity/v3/projects/4ca92f8b7d6c40839abf33f31d7d55f5/users/0030134573474da397006175a79243a8/roles/ea4c814713b149108e488de855b025e4 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 04:55:08.661236 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51d3c55-3e8a-4e9f-9b13-57410748f4b7 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:55:08.667490 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51d3c55-3e8a-4e9f-9b13-57410748f4b7 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:55:08.667490 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51d3c55-3e8a-4e9f-9b13-57410748f4b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:55:08.691680 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-a51d3c55-3e8a-4e9f-9b13-57410748f4b7 None None] MFA Rules not processed for user `0030134573474da397006175a79243a8`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:55:08.702299 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-a51d3c55-3e8a-4e9f-9b13-57410748f4b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:55:08.704963 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:55:08.764958 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1420/2830] 10.222.0.30 () {58 vars in 958 bytes} [Tue Sep 6 04:55:08 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 04:55:09.323571 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe6f3ea-137d-4b26-a28e-5b8d38f51748 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:55:09.325376 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-dbe6f3ea-137d-4b26-a28e-5b8d38f51748 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:55:09.381542 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:55:09.399790 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe6f3ea-137d-4b26-a28e-5b8d38f51748 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:55:09.405329 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15d0be1-9293-4323-893d-914bcb50315e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d4f2e4dbd41441d84f03ddb30191e7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b707733ce65c42edbc1fd95ad0a1c06c', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:55:09.405939 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15d0be1-9293-4323-893d-914bcb50315e service placement] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:55:09.406220 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15d0be1-9293-4323-893d-914bcb50315e service placement] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:55:09.406502 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15d0be1-9293-4323-893d-914bcb50315e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:55:09.408239 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-e15d0be1-9293-4323-893d-914bcb50315e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:55:09.420279 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15d0be1-9293-4323-893d-914bcb50315e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:55:09.421944 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15d0be1-9293-4323-893d-914bcb50315e service placement] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:55:09.432583 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1411/2831] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Sep 6 04:55:09 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 04:55:10.076473 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51d3c55-3e8a-4e9f-9b13-57410748f4b7 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:55:10.150644 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51d3c55-3e8a-4e9f-9b13-57410748f4b7 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:55:10.155423 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404c2b02-85c6-4465-8c35-88954199e646 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:55:10.156049 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404c2b02-85c6-4465-8c35-88954199e646 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:55:10.156352 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404c2b02-85c6-4465-8c35-88954199e646 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:55:10.156627 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404c2b02-85c6-4465-8c35-88954199e646 admin admin] PATH_INFO: `/v3/users/412b8a91889249b196cfe6daac1bcf20` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:55:10.157759 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404c2b02-85c6-4465-8c35-88954199e646 admin admin] RBAC: Authorizing `identity:delete_user(user_id=412b8a91889249b196cfe6daac1bcf20)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:55:10.159937 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404c2b02-85c6-4465-8c35-88954199e646 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:55:10.322720 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-404c2b02-85c6-4465-8c35-88954199e646 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '412b8a91889249b196cfe6daac1bcf20'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:55:10.402393 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-404c2b02-85c6-4465-8c35-88954199e646 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '412b8a91889249b196cfe6daac1bcf20'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:55:10.403006 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-404c2b02-85c6-4465-8c35-88954199e646 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '412b8a91889249b196cfe6daac1bcf20'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:55:10.417535 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-404c2b02-85c6-4465-8c35-88954199e646 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '412b8a91889249b196cfe6daac1bcf20'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:55:10.424691 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-404c2b02-85c6-4465-8c35-88954199e646 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '412b8a91889249b196cfe6daac1bcf20'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:55:10.430935 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1421/2832] 10.222.0.30 () {58 vars in 1224 bytes} [Tue Sep 6 04:55:10 2022] DELETE /identity/v3/users/412b8a91889249b196cfe6daac1bcf20 => generated 0 bytes in 355 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 04:55:10.672770 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15d0be1-9293-4323-893d-914bcb50315e service placement] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:55:10.686690 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15d0be1-9293-4323-893d-914bcb50315e service placement] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:55:10.744357 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46e411b-0be7-4cfc-93a4-d5c58ce347fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:55:10.745145 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46e411b-0be7-4cfc-93a4-d5c58ce347fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:55:10.745462 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46e411b-0be7-4cfc-93a4-d5c58ce347fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:55:10.745737 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46e411b-0be7-4cfc-93a4-d5c58ce347fc admin admin] PATH_INFO: `/v3/projects/2e29862416e748bdb5322ee789510ea5` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:55:10.746847 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e46e411b-0be7-4cfc-93a4-d5c58ce347fc admin admin] RBAC: Authorizing `identity:delete_project(project_id=2e29862416e748bdb5322ee789510ea5)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:55:10.748728 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e46e411b-0be7-4cfc-93a4-d5c58ce347fc admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:55:10.864737 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-e46e411b-0be7-4cfc-93a4-d5c58ce347fc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2e29862416e748bdb5322ee789510ea5', 'request_id': 'req-e46e411b-0be7-4cfc-93a4-d5c58ce347fc'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:55:10.943489 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-e46e411b-0be7-4cfc-93a4-d5c58ce347fc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2e29862416e748bdb5322ee789510ea5', 'request_id': 'req-e46e411b-0be7-4cfc-93a4-d5c58ce347fc'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:55:10.949986 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-e46e411b-0be7-4cfc-93a4-d5c58ce347fc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2e29862416e748bdb5322ee789510ea5', 'request_id': 'req-e46e411b-0be7-4cfc-93a4-d5c58ce347fc'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:55:10.960228 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-e46e411b-0be7-4cfc-93a4-d5c58ce347fc admin admin] The token cache is being invalidate because project 2e29862416e748bdb5322ee789510ea5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81697) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 04:55:10.965161 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-e46e411b-0be7-4cfc-93a4-d5c58ce347fc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:55:10.965161 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-e46e411b-0be7-4cfc-93a4-d5c58ce347fc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:55:10.965161 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1412/2833] 10.222.0.30 () {58 vars in 1233 bytes} [Tue Sep 6 04:55:10 2022] DELETE /identity/v3/projects/2e29862416e748bdb5322ee789510ea5 => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 04:55:10.967238 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404c2b02-85c6-4465-8c35-88954199e646 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:55:10.968628 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-404c2b02-85c6-4465-8c35-88954199e646 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:55:10.982370 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404c2b02-85c6-4465-8c35-88954199e646 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:55:11.008252 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:55:11.037911 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880bfb3b-072a-4a08-9546-0973e31383fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:55:11.038529 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880bfb3b-072a-4a08-9546-0973e31383fd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:55:11.038807 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880bfb3b-072a-4a08-9546-0973e31383fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:55:11.039078 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880bfb3b-072a-4a08-9546-0973e31383fd admin admin] PATH_INFO: `/v3/users/0030134573474da397006175a79243a8` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:55:11.040190 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880bfb3b-072a-4a08-9546-0973e31383fd admin admin] RBAC: Authorizing `identity:delete_user(user_id=0030134573474da397006175a79243a8)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:55:11.042816 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880bfb3b-072a-4a08-9546-0973e31383fd admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:55:11.221566 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-880bfb3b-072a-4a08-9546-0973e31383fd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0030134573474da397006175a79243a8'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:55:11.281874 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-880bfb3b-072a-4a08-9546-0973e31383fd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0030134573474da397006175a79243a8'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:55:11.282482 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-880bfb3b-072a-4a08-9546-0973e31383fd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0030134573474da397006175a79243a8'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:55:11.295078 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-880bfb3b-072a-4a08-9546-0973e31383fd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0030134573474da397006175a79243a8'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:55:11.301927 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-880bfb3b-072a-4a08-9546-0973e31383fd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0030134573474da397006175a79243a8'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:55:11.308014 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1422/2834] 10.222.0.30 () {58 vars in 1224 bytes} [Tue Sep 6 04:55:10 2022] DELETE /identity/v3/users/0030134573474da397006175a79243a8 => generated 0 bytes in 341 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 04:55:11.530365 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46e411b-0be7-4cfc-93a4-d5c58ce347fc admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:55:11.531838 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-e46e411b-0be7-4cfc-93a4-d5c58ce347fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:55:11.545517 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46e411b-0be7-4cfc-93a4-d5c58ce347fc admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:55:11.603998 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb372194-f5e5-4d34-ba65-2242d3d9a655 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:55:11.604651 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb372194-f5e5-4d34-ba65-2242d3d9a655 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:55:11.604932 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb372194-f5e5-4d34-ba65-2242d3d9a655 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:55:11.605204 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb372194-f5e5-4d34-ba65-2242d3d9a655 admin admin] PATH_INFO: `/v3/projects/4ca92f8b7d6c40839abf33f31d7d55f5` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:55:11.606355 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb372194-f5e5-4d34-ba65-2242d3d9a655 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4ca92f8b7d6c40839abf33f31d7d55f5)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:55:11.608248 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb372194-f5e5-4d34-ba65-2242d3d9a655 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:55:11.625102 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:55:11.721754 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-bb372194-f5e5-4d34-ba65-2242d3d9a655 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4ca92f8b7d6c40839abf33f31d7d55f5', 'request_id': 'req-bb372194-f5e5-4d34-ba65-2242d3d9a655'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:55:11.796582 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-bb372194-f5e5-4d34-ba65-2242d3d9a655 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4ca92f8b7d6c40839abf33f31d7d55f5', 'request_id': 'req-bb372194-f5e5-4d34-ba65-2242d3d9a655'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:55:11.803155 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-bb372194-f5e5-4d34-ba65-2242d3d9a655 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4ca92f8b7d6c40839abf33f31d7d55f5', 'request_id': 'req-bb372194-f5e5-4d34-ba65-2242d3d9a655'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:55:11.814312 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-bb372194-f5e5-4d34-ba65-2242d3d9a655 admin admin] The token cache is being invalidate because project 4ca92f8b7d6c40839abf33f31d7d55f5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81697) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 04:55:11.814602 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-bb372194-f5e5-4d34-ba65-2242d3d9a655 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:55:11.815259 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-bb372194-f5e5-4d34-ba65-2242d3d9a655 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:55:11.816727 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1413/2835] 10.222.0.30 () {58 vars in 1233 bytes} [Tue Sep 6 04:55:11 2022] DELETE /identity/v3/projects/4ca92f8b7d6c40839abf33f31d7d55f5 => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 04:55:11.822242 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94212155-5b49-4562-bb79-d8303df58ff1 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:55:11.822521 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94212155-5b49-4562-bb79-d8303df58ff1 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:55:11.822836 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94212155-5b49-4562-bb79-d8303df58ff1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:55:11.842193 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-94212155-5b49-4562-bb79-d8303df58ff1 None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:55:11.843377 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-94212155-5b49-4562-bb79-d8303df58ff1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:55:11.903052 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1423/2836] 10.222.0.30 () {58 vars in 958 bytes} [Tue Sep 6 04:55:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 04:55:11.911069 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e3c038-9316-46e6-bddb-ccf3a3621f2b None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:55:11.911349 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e3c038-9316-46e6-bddb-ccf3a3621f2b None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:55:11.911621 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e3c038-9316-46e6-bddb-ccf3a3621f2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:55:11.931649 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-d3e3c038-9316-46e6-bddb-ccf3a3621f2b None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:55:11.932834 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-d3e3c038-9316-46e6-bddb-ccf3a3621f2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:55:11.940604 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1414/2837] 10.222.0.30 () {58 vars in 958 bytes} [Tue Sep 6 04:55:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 04:55:11.944677 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94212155-5b49-4562-bb79-d8303df58ff1 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:55:11.955318 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94212155-5b49-4562-bb79-d8303df58ff1 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:55:11.959971 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a7d21a-364a-4380-a080-39cd2b4c9a2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:55:11.960572 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a7d21a-364a-4380-a080-39cd2b4c9a2d admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:55:11.960856 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a7d21a-364a-4380-a080-39cd2b4c9a2d admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:55:11.961131 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a7d21a-364a-4380-a080-39cd2b4c9a2d admin admin] PATH_INFO: `/v3/domains` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:55:11.961819 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a7d21a-364a-4380-a080-39cd2b4c9a2d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:55:11.962206 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a7d21a-364a-4380-a080-39cd2b4c9a2d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:55:11.963874 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a7d21a-364a-4380-a080-39cd2b4c9a2d admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:55:11.975222 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1424/2838] 10.222.0.30 () {58 vars in 1153 bytes} [Tue Sep 6 04:55:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:55:11.979413 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e3c038-9316-46e6-bddb-ccf3a3621f2b None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:55:11.988492 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e3c038-9316-46e6-bddb-ccf3a3621f2b None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:55:11.993037 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748554fc-aba6-44a1-8d26-b7d577417743 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:55:11.994197 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748554fc-aba6-44a1-8d26-b7d577417743 admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:55:11.994465 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748554fc-aba6-44a1-8d26-b7d577417743 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:55:11.994733 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748554fc-aba6-44a1-8d26-b7d577417743 admin admin] PATH_INFO: `/v3/projects` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:55:11.995366 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748554fc-aba6-44a1-8d26-b7d577417743 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:55:11.997236 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748554fc-aba6-44a1-8d26-b7d577417743 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:55:12.077840 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1415/2839] 10.222.0.30 () {60 vars in 1153 bytes} [Tue Sep 6 04:55:11 2022] POST /identity/v3/projects => generated 373 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 04:55:12.081939 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a7d21a-364a-4380-a080-39cd2b4c9a2d admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:55:12.092713 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a7d21a-364a-4380-a080-39cd2b4c9a2d admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:55:12.126367 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:55:12.153538 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39d2889-2d3c-4256-8677-2d7d71668c97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:55:12.154221 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39d2889-2d3c-4256-8677-2d7d71668c97 admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:55:12.154485 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39d2889-2d3c-4256-8677-2d7d71668c97 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:55:12.154757 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39d2889-2d3c-4256-8677-2d7d71668c97 admin admin] PATH_INFO: `/v3/users` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:55:12.155404 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39d2889-2d3c-4256-8677-2d7d71668c97 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:55:12.157323 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39d2889-2d3c-4256-8677-2d7d71668c97 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:55:12.248600 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1425/2840] 10.222.0.30 () {60 vars in 1144 bytes} [Tue Sep 6 04:55:12 2022] POST /identity/v3/users => generated 287 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 04:55:12.252765 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748554fc-aba6-44a1-8d26-b7d577417743 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:55:12.263562 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748554fc-aba6-44a1-8d26-b7d577417743 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:55:12.268393 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29f609e-10aa-4173-9f0a-3e5f02639c85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:55:12.269035 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29f609e-10aa-4173-9f0a-3e5f02639c85 admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:55:12.269399 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29f609e-10aa-4173-9f0a-3e5f02639c85 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:55:12.269684 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29f609e-10aa-4173-9f0a-3e5f02639c85 admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:55:12.270325 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29f609e-10aa-4173-9f0a-3e5f02639c85 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:55:12.271913 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29f609e-10aa-4173-9f0a-3e5f02639c85 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:55:12.282044 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1416/2841] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Sep 6 04:55:12 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 04:55:12.286309 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39d2889-2d3c-4256-8677-2d7d71668c97 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:55:12.297229 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39d2889-2d3c-4256-8677-2d7d71668c97 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:55:12.301873 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83208cf-2e45-41b2-b8b4-c50bef384156 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:55:12.302470 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83208cf-2e45-41b2-b8b4-c50bef384156 admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:55:12.302751 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83208cf-2e45-41b2-b8b4-c50bef384156 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:55:12.303022 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83208cf-2e45-41b2-b8b4-c50bef384156 admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:55:12.303658 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d83208cf-2e45-41b2-b8b4-c50bef384156 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:55:12.305244 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d83208cf-2e45-41b2-b8b4-c50bef384156 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:55:12.316233 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1426/2842] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Sep 6 04:55:12 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 04:55:12.320261 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29f609e-10aa-4173-9f0a-3e5f02639c85 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:55:12.330353 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29f609e-10aa-4173-9f0a-3e5f02639c85 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:55:12.334931 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f67c8e3-6c7d-4d95-893d-f7f44436d55f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:55:12.335605 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f67c8e3-6c7d-4d95-893d-f7f44436d55f admin admin] REQUEST_METHOD: `PUT` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:55:12.335886 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f67c8e3-6c7d-4d95-893d-f7f44436d55f admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:55:12.336155 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f67c8e3-6c7d-4d95-893d-f7f44436d55f admin admin] PATH_INFO: `/v3/projects/50d5ff73cfd44c6e86d482a60ec18ff8/users/0548908b5ca344a0a07ed7ed75cdc849/roles/29607b2de39f40d68bb6ceef64b8a732` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:55:12.350877 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f67c8e3-6c7d-4d95-893d-f7f44436d55f admin admin] RBAC: Authorizing `identity:create_grant(project_id=50d5ff73cfd44c6e86d482a60ec18ff8, user_id=0548908b5ca344a0a07ed7ed75cdc849, role_id=29607b2de39f40d68bb6ceef64b8a732)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:55:12.354631 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f67c8e3-6c7d-4d95-893d-f7f44436d55f admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:55:12.414105 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1417/2843] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Sep 6 04:55:12 2022] PUT /identity/v3/projects/50d5ff73cfd44c6e86d482a60ec18ff8/users/0548908b5ca344a0a07ed7ed75cdc849/roles/29607b2de39f40d68bb6ceef64b8a732 => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 04:55:12.424909 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a8da53-2ea5-4f9f-aea7-cf74b04f8793 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:55:12.425210 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a8da53-2ea5-4f9f-aea7-cf74b04f8793 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:55:12.425511 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a8da53-2ea5-4f9f-aea7-cf74b04f8793 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:55:12.452472 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-c8a8da53-2ea5-4f9f-aea7-cf74b04f8793 None None] MFA Rules not processed for user `0548908b5ca344a0a07ed7ed75cdc849`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:55:12.453709 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-c8a8da53-2ea5-4f9f-aea7-cf74b04f8793 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:55:12.518239 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1427/2844] 10.222.0.30 () {58 vars in 958 bytes} [Tue Sep 6 04:55:12 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 04:55:12.522732 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f67c8e3-6c7d-4d95-893d-f7f44436d55f admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:55:12.583444 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f67c8e3-6c7d-4d95-893d-f7f44436d55f admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:55:12.588492 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa45ded9-aefa-482c-bdd8-1815cf14da03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:55:12.589093 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa45ded9-aefa-482c-bdd8-1815cf14da03 admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:55:12.589401 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa45ded9-aefa-482c-bdd8-1815cf14da03 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:55:12.589676 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa45ded9-aefa-482c-bdd8-1815cf14da03 admin admin] PATH_INFO: `/v3/projects` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:55:12.590307 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa45ded9-aefa-482c-bdd8-1815cf14da03 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:55:12.592197 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa45ded9-aefa-482c-bdd8-1815cf14da03 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:55:12.710479 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1418/2845] 10.222.0.30 () {60 vars in 1153 bytes} [Tue Sep 6 04:55:12 2022] POST /identity/v3/projects => generated 373 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 04:55:12.715426 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a8da53-2ea5-4f9f-aea7-cf74b04f8793 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:55:12.726290 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a8da53-2ea5-4f9f-aea7-cf74b04f8793 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:55:12.782897 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f46f0f9-d421-4107-962b-c13bc02a74a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:55:12.783524 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f46f0f9-d421-4107-962b-c13bc02a74a9 admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:55:12.783803 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f46f0f9-d421-4107-962b-c13bc02a74a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:55:12.784326 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f46f0f9-d421-4107-962b-c13bc02a74a9 admin admin] PATH_INFO: `/v3/users` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:55:12.784958 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f46f0f9-d421-4107-962b-c13bc02a74a9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:55:12.786948 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f46f0f9-d421-4107-962b-c13bc02a74a9 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:55:12.870849 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1428/2846] 10.222.0.30 () {60 vars in 1144 bytes} [Tue Sep 6 04:55:12 2022] POST /identity/v3/users => generated 293 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 04:55:12.874723 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa45ded9-aefa-482c-bdd8-1815cf14da03 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:55:12.885470 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa45ded9-aefa-482c-bdd8-1815cf14da03 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:55:12.890312 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f60ed2-938f-42f8-989d-ab352d69223c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:55:12.890915 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f60ed2-938f-42f8-989d-ab352d69223c admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:55:12.891195 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f60ed2-938f-42f8-989d-ab352d69223c admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:55:12.891545 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f60ed2-938f-42f8-989d-ab352d69223c admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:55:12.892187 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f60ed2-938f-42f8-989d-ab352d69223c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:55:12.893815 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f60ed2-938f-42f8-989d-ab352d69223c admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:55:12.905892 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1419/2847] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Sep 6 04:55:12 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 04:55:12.910216 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f46f0f9-d421-4107-962b-c13bc02a74a9 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:55:12.921083 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f46f0f9-d421-4107-962b-c13bc02a74a9 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:55:12.926013 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6663723-314f-4ff3-87fc-8609500150ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:55:12.926661 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6663723-314f-4ff3-87fc-8609500150ed admin admin] REQUEST_METHOD: `PUT` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:55:12.926941 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6663723-314f-4ff3-87fc-8609500150ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:55:12.927209 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6663723-314f-4ff3-87fc-8609500150ed admin admin] PATH_INFO: `/v3/projects/afa5c36b40194223848845f1a2300c09/users/6fe70c220cc844e686034030a7cf1f4e/roles/ea4c814713b149108e488de855b025e4` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:55:12.942193 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6663723-314f-4ff3-87fc-8609500150ed admin admin] RBAC: Authorizing `identity:create_grant(project_id=afa5c36b40194223848845f1a2300c09, user_id=6fe70c220cc844e686034030a7cf1f4e, role_id=ea4c814713b149108e488de855b025e4)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:55:12.944856 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6663723-314f-4ff3-87fc-8609500150ed admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:55:12.996019 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1429/2848] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Sep 6 04:55:12 2022] PUT /identity/v3/projects/afa5c36b40194223848845f1a2300c09/users/6fe70c220cc844e686034030a7cf1f4e/roles/ea4c814713b149108e488de855b025e4 => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 04:55:13.000170 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f60ed2-938f-42f8-989d-ab352d69223c admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:55:13.011305 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f60ed2-938f-42f8-989d-ab352d69223c admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:55:13.020006 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:55:13.067435 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202d71bd-7098-4eef-9fde-27562e8123b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:55:13.068080 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202d71bd-7098-4eef-9fde-27562e8123b4 admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:55:13.068355 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202d71bd-7098-4eef-9fde-27562e8123b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:55:13.068626 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202d71bd-7098-4eef-9fde-27562e8123b4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:55:13.069255 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202d71bd-7098-4eef-9fde-27562e8123b4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:55:13.070861 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202d71bd-7098-4eef-9fde-27562e8123b4 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:55:13.081276 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1420/2849] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Sep 6 04:55:12 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 04:55:13.085580 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6663723-314f-4ff3-87fc-8609500150ed admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:55:13.099740 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6663723-314f-4ff3-87fc-8609500150ed admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:55:13.104671 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9dd07b-56ee-4962-b933-3fa1b1ab6825 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:55:13.105346 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9dd07b-56ee-4962-b933-3fa1b1ab6825 admin admin] REQUEST_METHOD: `PUT` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:55:13.105626 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9dd07b-56ee-4962-b933-3fa1b1ab6825 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:55:13.105892 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9dd07b-56ee-4962-b933-3fa1b1ab6825 admin admin] PATH_INFO: `/v3/projects/afa5c36b40194223848845f1a2300c09/users/6fe70c220cc844e686034030a7cf1f4e/roles/ea4c814713b149108e488de855b025e4` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:55:13.107789 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9dd07b-56ee-4962-b933-3fa1b1ab6825 admin admin] RBAC: Authorizing `identity:create_grant(project_id=afa5c36b40194223848845f1a2300c09, user_id=6fe70c220cc844e686034030a7cf1f4e, role_id=ea4c814713b149108e488de855b025e4)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:55:13.110430 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9dd07b-56ee-4962-b933-3fa1b1ab6825 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:55:13.113498 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:55:13.125107 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1430/2850] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Sep 6 04:55:13 2022] PUT /identity/v3/projects/afa5c36b40194223848845f1a2300c09/users/6fe70c220cc844e686034030a7cf1f4e/roles/ea4c814713b149108e488de855b025e4 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 04:55:13.186487 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30784a52-e248-41e4-b703-8cd20f240b90 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:55:13.186775 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30784a52-e248-41e4-b703-8cd20f240b90 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:55:13.187052 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30784a52-e248-41e4-b703-8cd20f240b90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:55:13.215618 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-30784a52-e248-41e4-b703-8cd20f240b90 None None] MFA Rules not processed for user `6fe70c220cc844e686034030a7cf1f4e`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:55:13.216838 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-30784a52-e248-41e4-b703-8cd20f240b90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:55:13.284635 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1421/2851] 10.222.0.30 () {58 vars in 958 bytes} [Tue Sep 6 04:55:13 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 04:55:13.295979 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9dd07b-56ee-4962-b933-3fa1b1ab6825 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:55:13.297735 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-2f9dd07b-56ee-4962-b933-3fa1b1ab6825 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:55:13.376001 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9dd07b-56ee-4962-b933-3fa1b1ab6825 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:55:13.379653 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92bbdf2d-126a-458e-8d8e-bfbd9da64348 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd09b5ab675634a2eb4b03755743f7d0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b707733ce65c42edbc1fd95ad0a1c06c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:55:13.380266 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bbdf2d-126a-458e-8d8e-bfbd9da64348 service cinder] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:55:13.380549 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bbdf2d-126a-458e-8d8e-bfbd9da64348 service cinder] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:55:13.380822 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bbdf2d-126a-458e-8d8e-bfbd9da64348 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:55:13.390565 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92bbdf2d-126a-458e-8d8e-bfbd9da64348 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:55:13.392004 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92bbdf2d-126a-458e-8d8e-bfbd9da64348 service cinder] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:55:13.401380 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1431/2852] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Sep 6 04:55:13 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 06 04:55:13.588875 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30784a52-e248-41e4-b703-8cd20f240b90 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:55:13.598523 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30784a52-e248-41e4-b703-8cd20f240b90 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:55:13.602031 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f91b4fe-ecb6-454d-a5ef-87e624066aaf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd09b5ab675634a2eb4b03755743f7d0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b707733ce65c42edbc1fd95ad0a1c06c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:55:13.606089 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f91b4fe-ecb6-454d-a5ef-87e624066aaf service cinder] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:55:13.606089 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f91b4fe-ecb6-454d-a5ef-87e624066aaf service cinder] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:55:13.606089 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f91b4fe-ecb6-454d-a5ef-87e624066aaf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:55:13.658600 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f91b4fe-ecb6-454d-a5ef-87e624066aaf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:55:13.660229 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f91b4fe-ecb6-454d-a5ef-87e624066aaf service cinder] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:55:13.669868 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1422/2853] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Sep 6 04:55:13 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 04:55:23.397129 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=37 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:55:23.664986 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=85 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:56:05.874518 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92bbdf2d-126a-458e-8d8e-bfbd9da64348 service cinder] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:56:05.938276 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92bbdf2d-126a-458e-8d8e-bfbd9da64348 service cinder] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:56:05.942881 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2df08fa-1010-4a09-bc3e-af8c2ec5965c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:56:05.948903 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2df08fa-1010-4a09-bc3e-af8c2ec5965c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:56:05.948903 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2df08fa-1010-4a09-bc3e-af8c2ec5965c admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:56:05.948903 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2df08fa-1010-4a09-bc3e-af8c2ec5965c admin admin] PATH_INFO: `/v3/users/0548908b5ca344a0a07ed7ed75cdc849` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:56:05.948903 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2df08fa-1010-4a09-bc3e-af8c2ec5965c admin admin] RBAC: Authorizing `identity:delete_user(user_id=0548908b5ca344a0a07ed7ed75cdc849)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:56:05.948903 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2df08fa-1010-4a09-bc3e-af8c2ec5965c admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:56:06.129968 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-d2df08fa-1010-4a09-bc3e-af8c2ec5965c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0548908b5ca344a0a07ed7ed75cdc849'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:56:06.212728 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-d2df08fa-1010-4a09-bc3e-af8c2ec5965c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0548908b5ca344a0a07ed7ed75cdc849'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:56:06.213280 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-d2df08fa-1010-4a09-bc3e-af8c2ec5965c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0548908b5ca344a0a07ed7ed75cdc849'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:56:06.226006 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-d2df08fa-1010-4a09-bc3e-af8c2ec5965c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0548908b5ca344a0a07ed7ed75cdc849'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:56:06.233428 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-d2df08fa-1010-4a09-bc3e-af8c2ec5965c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0548908b5ca344a0a07ed7ed75cdc849'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:56:06.239388 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1432/2854] 10.222.0.30 () {58 vars in 1224 bytes} [Tue Sep 6 04:56:05 2022] DELETE /identity/v3/users/0548908b5ca344a0a07ed7ed75cdc849 => generated 0 bytes in 366 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 04:56:06.243978 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669bab50-b1c2-4acd-8467-adc39990b98f None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:56:06.244458 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669bab50-b1c2-4acd-8467-adc39990b98f None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:56:06.244569 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669bab50-b1c2-4acd-8467-adc39990b98f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:56:06.263741 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-669bab50-b1c2-4acd-8467-adc39990b98f None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:56:06.265570 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-669bab50-b1c2-4acd-8467-adc39990b98f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:56:06.322812 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1423/2855] 10.222.0.30 () {58 vars in 958 bytes} [Tue Sep 6 04:56:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 04:56:06.333455 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2df08fa-1010-4a09-bc3e-af8c2ec5965c admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:56:06.334854 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-d2df08fa-1010-4a09-bc3e-af8c2ec5965c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:56:06.395751 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2df08fa-1010-4a09-bc3e-af8c2ec5965c admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:56:06.399570 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6f5528-23e3-4472-bd63-46e33a70d995 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90492440f6a6471e8392361c098428aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b707733ce65c42edbc1fd95ad0a1c06c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:56:06.400183 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6f5528-23e3-4472-bd63-46e33a70d995 service neutron] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:56:06.400459 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6f5528-23e3-4472-bd63-46e33a70d995 service neutron] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:56:06.400730 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6f5528-23e3-4472-bd63-46e33a70d995 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:56:06.410077 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6f5528-23e3-4472-bd63-46e33a70d995 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:56:06.411512 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6f5528-23e3-4472-bd63-46e33a70d995 service neutron] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:56:06.421721 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1433/2856] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Sep 6 04:56:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 04:56:06.814923 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669bab50-b1c2-4acd-8467-adc39990b98f None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:56:06.816383 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-669bab50-b1c2-4acd-8467-adc39990b98f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:56:06.828349 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669bab50-b1c2-4acd-8467-adc39990b98f None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:56:06.833434 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2641ba9f-1257-44b9-8b5f-2a79c41ebd3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:56:06.834078 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2641ba9f-1257-44b9-8b5f-2a79c41ebd3b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:56:06.834412 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2641ba9f-1257-44b9-8b5f-2a79c41ebd3b admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:56:06.834696 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2641ba9f-1257-44b9-8b5f-2a79c41ebd3b admin admin] PATH_INFO: `/v3/projects/50d5ff73cfd44c6e86d482a60ec18ff8` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:56:06.835829 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2641ba9f-1257-44b9-8b5f-2a79c41ebd3b admin admin] RBAC: Authorizing `identity:delete_project(project_id=50d5ff73cfd44c6e86d482a60ec18ff8)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:56:06.837768 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2641ba9f-1257-44b9-8b5f-2a79c41ebd3b admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:56:06.961739 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-2641ba9f-1257-44b9-8b5f-2a79c41ebd3b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '50d5ff73cfd44c6e86d482a60ec18ff8', 'request_id': 'req-2641ba9f-1257-44b9-8b5f-2a79c41ebd3b'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:56:07.046696 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-2641ba9f-1257-44b9-8b5f-2a79c41ebd3b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '50d5ff73cfd44c6e86d482a60ec18ff8', 'request_id': 'req-2641ba9f-1257-44b9-8b5f-2a79c41ebd3b'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:56:07.053187 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-2641ba9f-1257-44b9-8b5f-2a79c41ebd3b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '50d5ff73cfd44c6e86d482a60ec18ff8', 'request_id': 'req-2641ba9f-1257-44b9-8b5f-2a79c41ebd3b'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:56:07.064039 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-2641ba9f-1257-44b9-8b5f-2a79c41ebd3b admin admin] The token cache is being invalidate because project 50d5ff73cfd44c6e86d482a60ec18ff8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81697) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 04:56:07.064266 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-2641ba9f-1257-44b9-8b5f-2a79c41ebd3b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:56:07.064887 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-2641ba9f-1257-44b9-8b5f-2a79c41ebd3b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:56:07.066388 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1424/2857] 10.222.0.30 () {58 vars in 1233 bytes} [Tue Sep 6 04:56:06 2022] DELETE /identity/v3/projects/50d5ff73cfd44c6e86d482a60ec18ff8 => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 04:56:07.070506 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6f5528-23e3-4472-bd63-46e33a70d995 service neutron] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:56:07.080982 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6f5528-23e3-4472-bd63-46e33a70d995 service neutron] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:56:07.098974 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:56:07.136548 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abfa4b35-6c88-4347-a44a-af4f60ed23ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:56:07.137172 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfa4b35-6c88-4347-a44a-af4f60ed23ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:56:07.137487 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfa4b35-6c88-4347-a44a-af4f60ed23ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:56:07.137762 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfa4b35-6c88-4347-a44a-af4f60ed23ab admin admin] PATH_INFO: `/v3/users/6fe70c220cc844e686034030a7cf1f4e` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:56:07.139669 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abfa4b35-6c88-4347-a44a-af4f60ed23ab admin admin] RBAC: Authorizing `identity:delete_user(user_id=6fe70c220cc844e686034030a7cf1f4e)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:56:07.141632 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abfa4b35-6c88-4347-a44a-af4f60ed23ab admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:56:07.389502 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-abfa4b35-6c88-4347-a44a-af4f60ed23ab admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6fe70c220cc844e686034030a7cf1f4e'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:56:07.458225 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-abfa4b35-6c88-4347-a44a-af4f60ed23ab admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6fe70c220cc844e686034030a7cf1f4e'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:56:07.458829 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-abfa4b35-6c88-4347-a44a-af4f60ed23ab admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6fe70c220cc844e686034030a7cf1f4e'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:56:07.472252 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-abfa4b35-6c88-4347-a44a-af4f60ed23ab admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6fe70c220cc844e686034030a7cf1f4e'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:56:07.479802 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-abfa4b35-6c88-4347-a44a-af4f60ed23ab admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6fe70c220cc844e686034030a7cf1f4e'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:56:07.486472 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1434/2858] 10.222.0.30 () {58 vars in 1224 bytes} [Tue Sep 6 04:56:07 2022] DELETE /identity/v3/users/6fe70c220cc844e686034030a7cf1f4e => generated 0 bytes in 416 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 04:56:07.894827 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2641ba9f-1257-44b9-8b5f-2a79c41ebd3b admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:56:07.897803 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-2641ba9f-1257-44b9-8b5f-2a79c41ebd3b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:56:07.910773 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2641ba9f-1257-44b9-8b5f-2a79c41ebd3b admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:56:07.971012 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244819fa-7714-46ef-a01c-0077297f671c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:56:07.971656 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244819fa-7714-46ef-a01c-0077297f671c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:56:07.971938 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244819fa-7714-46ef-a01c-0077297f671c admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:56:07.972214 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244819fa-7714-46ef-a01c-0077297f671c admin admin] PATH_INFO: `/v3/projects/afa5c36b40194223848845f1a2300c09` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:56:07.973389 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244819fa-7714-46ef-a01c-0077297f671c admin admin] RBAC: Authorizing `identity:delete_project(project_id=afa5c36b40194223848845f1a2300c09)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:56:07.975281 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244819fa-7714-46ef-a01c-0077297f671c admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:56:08.060096 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:56:08.075949 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-244819fa-7714-46ef-a01c-0077297f671c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'afa5c36b40194223848845f1a2300c09', 'request_id': 'req-244819fa-7714-46ef-a01c-0077297f671c'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:56:08.167293 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-244819fa-7714-46ef-a01c-0077297f671c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'afa5c36b40194223848845f1a2300c09', 'request_id': 'req-244819fa-7714-46ef-a01c-0077297f671c'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:56:08.173872 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-244819fa-7714-46ef-a01c-0077297f671c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'afa5c36b40194223848845f1a2300c09', 'request_id': 'req-244819fa-7714-46ef-a01c-0077297f671c'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:56:08.184898 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-244819fa-7714-46ef-a01c-0077297f671c admin admin] The token cache is being invalidate because project afa5c36b40194223848845f1a2300c09 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81697) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 04:56:08.185203 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-244819fa-7714-46ef-a01c-0077297f671c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:56:08.185846 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-244819fa-7714-46ef-a01c-0077297f671c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:56:08.187408 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1425/2859] 10.222.0.30 () {58 vars in 1233 bytes} [Tue Sep 6 04:56:07 2022] DELETE /identity/v3/projects/afa5c36b40194223848845f1a2300c09 => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 04:56:08.192227 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a164d5ad-7461-4735-9618-cc09385ff435 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:56:08.192689 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a164d5ad-7461-4735-9618-cc09385ff435 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:56:08.192999 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a164d5ad-7461-4735-9618-cc09385ff435 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:56:08.214492 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-a164d5ad-7461-4735-9618-cc09385ff435 None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:56:08.215673 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-a164d5ad-7461-4735-9618-cc09385ff435 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:56:08.274526 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1435/2860] 10.222.0.30 () {58 vars in 958 bytes} [Tue Sep 6 04:56:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 04:56:08.284663 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4aacc50-73dc-44af-b8a7-d7a63843556d None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:56:08.284952 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4aacc50-73dc-44af-b8a7-d7a63843556d None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:56:08.285229 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4aacc50-73dc-44af-b8a7-d7a63843556d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:56:08.305197 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-f4aacc50-73dc-44af-b8a7-d7a63843556d None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:56:08.306438 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-f4aacc50-73dc-44af-b8a7-d7a63843556d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:56:08.314958 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1426/2861] 10.222.0.30 () {58 vars in 958 bytes} [Tue Sep 6 04:56:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 04:56:08.319207 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a164d5ad-7461-4735-9618-cc09385ff435 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:56:08.329713 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a164d5ad-7461-4735-9618-cc09385ff435 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:56:08.334323 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341e8c46-e49d-4fd4-a5ea-0e1b6a1d7567 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:56:08.334927 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341e8c46-e49d-4fd4-a5ea-0e1b6a1d7567 admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:56:08.335210 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341e8c46-e49d-4fd4-a5ea-0e1b6a1d7567 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:56:08.335479 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341e8c46-e49d-4fd4-a5ea-0e1b6a1d7567 admin admin] PATH_INFO: `/v3/domains` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:56:08.336130 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341e8c46-e49d-4fd4-a5ea-0e1b6a1d7567 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81698) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:56:08.336456 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341e8c46-e49d-4fd4-a5ea-0e1b6a1d7567 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:56:08.338098 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341e8c46-e49d-4fd4-a5ea-0e1b6a1d7567 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:56:08.350098 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1436/2862] 10.222.0.30 () {58 vars in 1153 bytes} [Tue Sep 6 04:56:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:56:08.354408 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4aacc50-73dc-44af-b8a7-d7a63843556d None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:56:08.364233 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4aacc50-73dc-44af-b8a7-d7a63843556d None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:56:08.368900 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6365ea7-77e1-46cb-9d98-4f6247f983db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:56:08.369527 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6365ea7-77e1-46cb-9d98-4f6247f983db admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:56:08.369804 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6365ea7-77e1-46cb-9d98-4f6247f983db admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:56:08.370072 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6365ea7-77e1-46cb-9d98-4f6247f983db admin admin] PATH_INFO: `/v3/projects` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:56:08.370707 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6365ea7-77e1-46cb-9d98-4f6247f983db admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:56:08.372585 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6365ea7-77e1-46cb-9d98-4f6247f983db admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:56:08.454890 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1427/2863] 10.222.0.30 () {60 vars in 1153 bytes} [Tue Sep 6 04:56:08 2022] POST /identity/v3/projects => generated 381 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 04:56:08.459202 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341e8c46-e49d-4fd4-a5ea-0e1b6a1d7567 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:56:08.470065 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341e8c46-e49d-4fd4-a5ea-0e1b6a1d7567 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:56:08.497793 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:56:08.529880 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44fa689-ed1b-4946-a474-96c6f0e6f49e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:56:08.530482 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44fa689-ed1b-4946-a474-96c6f0e6f49e admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:56:08.530757 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44fa689-ed1b-4946-a474-96c6f0e6f49e admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:56:08.531029 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44fa689-ed1b-4946-a474-96c6f0e6f49e admin admin] PATH_INFO: `/v3/users` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:56:08.531655 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44fa689-ed1b-4946-a474-96c6f0e6f49e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:56:08.533634 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44fa689-ed1b-4946-a474-96c6f0e6f49e admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:56:08.633687 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1437/2864] 10.222.0.30 () {60 vars in 1144 bytes} [Tue Sep 6 04:56:08 2022] POST /identity/v3/users => generated 291 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 04:56:08.638002 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6365ea7-77e1-46cb-9d98-4f6247f983db admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:56:08.648616 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6365ea7-77e1-46cb-9d98-4f6247f983db admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:56:08.653256 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb41693c-06bf-44a9-9b4f-1f54d82198ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:56:08.653886 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb41693c-06bf-44a9-9b4f-1f54d82198ec admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:56:08.654163 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb41693c-06bf-44a9-9b4f-1f54d82198ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:56:08.655293 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb41693c-06bf-44a9-9b4f-1f54d82198ec admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:56:08.655652 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb41693c-06bf-44a9-9b4f-1f54d82198ec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:56:08.657280 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb41693c-06bf-44a9-9b4f-1f54d82198ec admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:56:08.667278 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1428/2865] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Sep 6 04:56:08 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 04:56:08.671933 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44fa689-ed1b-4946-a474-96c6f0e6f49e admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:56:08.681927 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44fa689-ed1b-4946-a474-96c6f0e6f49e admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:56:08.686808 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd303ce-88ee-4d93-aefd-fc73eebb0ef4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:56:08.687442 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd303ce-88ee-4d93-aefd-fc73eebb0ef4 admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:56:08.687723 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd303ce-88ee-4d93-aefd-fc73eebb0ef4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:56:08.687994 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd303ce-88ee-4d93-aefd-fc73eebb0ef4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:56:08.688637 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd303ce-88ee-4d93-aefd-fc73eebb0ef4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:56:08.690380 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd303ce-88ee-4d93-aefd-fc73eebb0ef4 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:56:08.700384 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1438/2866] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Sep 6 04:56:08 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 04:56:08.704717 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb41693c-06bf-44a9-9b4f-1f54d82198ec admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:56:08.714623 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb41693c-06bf-44a9-9b4f-1f54d82198ec admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:56:08.719780 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15151e2d-8610-44fd-8b22-f940296bf684 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:56:08.720423 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15151e2d-8610-44fd-8b22-f940296bf684 admin admin] REQUEST_METHOD: `PUT` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:56:08.720706 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15151e2d-8610-44fd-8b22-f940296bf684 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:56:08.720979 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15151e2d-8610-44fd-8b22-f940296bf684 admin admin] PATH_INFO: `/v3/projects/08628313d57e46be90123e547d651d33/users/45555e9de7e44aa497ca47583673b1c5/roles/29607b2de39f40d68bb6ceef64b8a732` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:56:08.736012 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15151e2d-8610-44fd-8b22-f940296bf684 admin admin] RBAC: Authorizing `identity:create_grant(project_id=08628313d57e46be90123e547d651d33, user_id=45555e9de7e44aa497ca47583673b1c5, role_id=29607b2de39f40d68bb6ceef64b8a732)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:56:08.738714 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15151e2d-8610-44fd-8b22-f940296bf684 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:56:08.807268 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1429/2867] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Sep 6 04:56:08 2022] PUT /identity/v3/projects/08628313d57e46be90123e547d651d33/users/45555e9de7e44aa497ca47583673b1c5/roles/29607b2de39f40d68bb6ceef64b8a732 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 04:56:08.817759 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbed6cdd-8160-419e-abaa-ccb4f5dfdc12 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:56:08.818042 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbed6cdd-8160-419e-abaa-ccb4f5dfdc12 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:56:08.818317 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbed6cdd-8160-419e-abaa-ccb4f5dfdc12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:56:08.845917 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-dbed6cdd-8160-419e-abaa-ccb4f5dfdc12 None None] MFA Rules not processed for user `45555e9de7e44aa497ca47583673b1c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:56:08.847104 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-dbed6cdd-8160-419e-abaa-ccb4f5dfdc12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:56:08.911522 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1439/2868] 10.222.0.30 () {58 vars in 958 bytes} [Tue Sep 6 04:56:08 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 04:56:08.916079 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15151e2d-8610-44fd-8b22-f940296bf684 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:56:08.978593 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15151e2d-8610-44fd-8b22-f940296bf684 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:56:08.983735 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c23989c-3247-432e-ac8c-6317e0782bbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:56:08.984342 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c23989c-3247-432e-ac8c-6317e0782bbb admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:56:08.984627 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c23989c-3247-432e-ac8c-6317e0782bbb admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:56:08.984904 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c23989c-3247-432e-ac8c-6317e0782bbb admin admin] PATH_INFO: `/v3/projects` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:56:08.985576 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c23989c-3247-432e-ac8c-6317e0782bbb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:56:08.987478 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c23989c-3247-432e-ac8c-6317e0782bbb admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:56:09.085896 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1430/2869] 10.222.0.30 () {60 vars in 1153 bytes} [Tue Sep 6 04:56:08 2022] POST /identity/v3/projects => generated 379 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 04:56:09.090003 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbed6cdd-8160-419e-abaa-ccb4f5dfdc12 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:56:09.100991 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbed6cdd-8160-419e-abaa-ccb4f5dfdc12 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:56:09.158874 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748cdc88-0df7-43f9-bee4-45dd7ea9a209 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:56:09.159732 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748cdc88-0df7-43f9-bee4-45dd7ea9a209 admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:56:09.160093 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748cdc88-0df7-43f9-bee4-45dd7ea9a209 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:56:09.160430 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748cdc88-0df7-43f9-bee4-45dd7ea9a209 admin admin] PATH_INFO: `/v3/users` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:56:09.161087 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748cdc88-0df7-43f9-bee4-45dd7ea9a209 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:56:09.163261 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748cdc88-0df7-43f9-bee4-45dd7ea9a209 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:56:09.251781 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1440/2870] 10.222.0.30 () {60 vars in 1144 bytes} [Tue Sep 6 04:56:09 2022] POST /identity/v3/users => generated 296 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 04:56:09.255890 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c23989c-3247-432e-ac8c-6317e0782bbb admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:56:09.267517 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c23989c-3247-432e-ac8c-6317e0782bbb admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:56:09.272138 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205d15f4-9485-412d-8f1d-6c6ad1175726 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:56:09.272769 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205d15f4-9485-412d-8f1d-6c6ad1175726 admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:56:09.273047 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205d15f4-9485-412d-8f1d-6c6ad1175726 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:56:09.273346 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205d15f4-9485-412d-8f1d-6c6ad1175726 admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:56:09.273985 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205d15f4-9485-412d-8f1d-6c6ad1175726 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:56:09.275579 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205d15f4-9485-412d-8f1d-6c6ad1175726 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:56:09.286940 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1431/2871] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Sep 6 04:56:09 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 04:56:09.290977 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748cdc88-0df7-43f9-bee4-45dd7ea9a209 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:56:09.301122 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748cdc88-0df7-43f9-bee4-45dd7ea9a209 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:56:09.305781 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89fabd11-6ef2-4e69-bebd-723a481a096e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:56:09.306422 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fabd11-6ef2-4e69-bebd-723a481a096e admin admin] REQUEST_METHOD: `PUT` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:56:09.306750 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fabd11-6ef2-4e69-bebd-723a481a096e admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:56:09.307020 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fabd11-6ef2-4e69-bebd-723a481a096e admin admin] PATH_INFO: `/v3/projects/7f721731017347838f1f1e86dd8b8481/users/7b61e9860501422ca23a1fe1572cc53f/roles/ea4c814713b149108e488de855b025e4` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:56:09.322329 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89fabd11-6ef2-4e69-bebd-723a481a096e admin admin] RBAC: Authorizing `identity:create_grant(project_id=7f721731017347838f1f1e86dd8b8481, user_id=7b61e9860501422ca23a1fe1572cc53f, role_id=ea4c814713b149108e488de855b025e4)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:56:09.324948 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89fabd11-6ef2-4e69-bebd-723a481a096e admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:56:09.423320 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1441/2872] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Sep 6 04:56:09 2022] PUT /identity/v3/projects/7f721731017347838f1f1e86dd8b8481/users/7b61e9860501422ca23a1fe1572cc53f/roles/ea4c814713b149108e488de855b025e4 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 04:56:09.427396 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205d15f4-9485-412d-8f1d-6c6ad1175726 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:56:09.438090 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205d15f4-9485-412d-8f1d-6c6ad1175726 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:56:09.459305 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:56:09.494964 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3746eee7-d8c4-4376-943e-2d516cd1295b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:56:09.495575 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3746eee7-d8c4-4376-943e-2d516cd1295b admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:56:09.495859 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3746eee7-d8c4-4376-943e-2d516cd1295b admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:56:09.496133 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3746eee7-d8c4-4376-943e-2d516cd1295b admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:56:09.496769 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3746eee7-d8c4-4376-943e-2d516cd1295b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:56:09.499032 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3746eee7-d8c4-4376-943e-2d516cd1295b admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:56:09.509933 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1432/2873] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Sep 6 04:56:09 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 04:56:09.513404 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89fabd11-6ef2-4e69-bebd-723a481a096e admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:56:09.519838 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:56:09.525088 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89fabd11-6ef2-4e69-bebd-723a481a096e admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:56:09.529742 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b041a24-86d7-48cb-b983-f404119b3a6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:56:09.530385 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b041a24-86d7-48cb-b983-f404119b3a6f admin admin] REQUEST_METHOD: `PUT` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:56:09.530660 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b041a24-86d7-48cb-b983-f404119b3a6f admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:56:09.530929 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b041a24-86d7-48cb-b983-f404119b3a6f admin admin] PATH_INFO: `/v3/projects/7f721731017347838f1f1e86dd8b8481/users/7b61e9860501422ca23a1fe1572cc53f/roles/ea4c814713b149108e488de855b025e4` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:56:09.532832 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b041a24-86d7-48cb-b983-f404119b3a6f admin admin] RBAC: Authorizing `identity:create_grant(project_id=7f721731017347838f1f1e86dd8b8481, user_id=7b61e9860501422ca23a1fe1572cc53f, role_id=ea4c814713b149108e488de855b025e4)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:56:09.535734 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b041a24-86d7-48cb-b983-f404119b3a6f admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:56:09.546621 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1442/2874] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Sep 6 04:56:09 2022] PUT /identity/v3/projects/7f721731017347838f1f1e86dd8b8481/users/7b61e9860501422ca23a1fe1572cc53f/roles/ea4c814713b149108e488de855b025e4 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 04:56:09.557384 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70dc8a8b-6342-47cb-9a38-cb6cfbac1d00 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:56:09.557680 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70dc8a8b-6342-47cb-9a38-cb6cfbac1d00 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:56:09.557963 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70dc8a8b-6342-47cb-9a38-cb6cfbac1d00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:56:09.586307 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-70dc8a8b-6342-47cb-9a38-cb6cfbac1d00 None None] MFA Rules not processed for user `7b61e9860501422ca23a1fe1572cc53f`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:56:09.587528 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-70dc8a8b-6342-47cb-9a38-cb6cfbac1d00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:56:09.655712 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1433/2875] 10.222.0.30 () {58 vars in 958 bytes} [Tue Sep 6 04:56:09 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 04:56:09.668056 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b041a24-86d7-48cb-b983-f404119b3a6f admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:56:09.669754 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-6b041a24-86d7-48cb-b983-f404119b3a6f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:56:09.727587 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b041a24-86d7-48cb-b983-f404119b3a6f admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:56:09.731317 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e866f8-0efd-46b9-ae40-c588dd462bef service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd09b5ab675634a2eb4b03755743f7d0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b707733ce65c42edbc1fd95ad0a1c06c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:56:09.731920 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e866f8-0efd-46b9-ae40-c588dd462bef service cinder] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:56:09.732202 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e866f8-0efd-46b9-ae40-c588dd462bef service cinder] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:56:09.732975 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e866f8-0efd-46b9-ae40-c588dd462bef service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:56:09.741923 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e866f8-0efd-46b9-ae40-c588dd462bef service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:56:09.743359 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e866f8-0efd-46b9-ae40-c588dd462bef service cinder] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:56:09.753447 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1443/2876] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Sep 6 04:56:09 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 04:56:10.743589 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70dc8a8b-6342-47cb-9a38-cb6cfbac1d00 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:56:10.753283 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70dc8a8b-6342-47cb-9a38-cb6cfbac1d00 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:56:10.757981 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea4578f-97f7-43f9-936c-dce7883857d2 tempest-VolumeQuotaClassesTest-526471283 tempest-VolumeQuotaClassesTest-526471283-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b61e9860501422ca23a1fe1572cc53f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7f721731017347838f1f1e86dd8b8481', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:56:10.759013 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea4578f-97f7-43f9-936c-dce7883857d2 tempest-VolumeQuotaClassesTest-526471283 tempest-VolumeQuotaClassesTest-526471283-project-admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:56:10.759497 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea4578f-97f7-43f9-936c-dce7883857d2 tempest-VolumeQuotaClassesTest-526471283 tempest-VolumeQuotaClassesTest-526471283-project-admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:56:10.760021 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea4578f-97f7-43f9-936c-dce7883857d2 tempest-VolumeQuotaClassesTest-526471283 tempest-VolumeQuotaClassesTest-526471283-project-admin] PATH_INFO: `/v3/domains` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:56:10.761219 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea4578f-97f7-43f9-936c-dce7883857d2 tempest-VolumeQuotaClassesTest-526471283 tempest-VolumeQuotaClassesTest-526471283-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:56:10.761790 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea4578f-97f7-43f9-936c-dce7883857d2 tempest-VolumeQuotaClassesTest-526471283 tempest-VolumeQuotaClassesTest-526471283-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:56:10.765094 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea4578f-97f7-43f9-936c-dce7883857d2 tempest-VolumeQuotaClassesTest-526471283 tempest-VolumeQuotaClassesTest-526471283-project-admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:56:10.782886 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1434/2877] 10.222.0.30 () {58 vars in 1153 bytes} [Tue Sep 6 04:56:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:56:10.787480 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e866f8-0efd-46b9-ae40-c588dd462bef service cinder] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:56:10.799499 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e866f8-0efd-46b9-ae40-c588dd462bef service cinder] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:56:10.804875 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ac671b-a86e-4a73-a4ee-d6747df77289 tempest-VolumeQuotaClassesTest-526471283 tempest-VolumeQuotaClassesTest-526471283-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b61e9860501422ca23a1fe1572cc53f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7f721731017347838f1f1e86dd8b8481', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:56:10.805526 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ac671b-a86e-4a73-a4ee-d6747df77289 tempest-VolumeQuotaClassesTest-526471283 tempest-VolumeQuotaClassesTest-526471283-project-admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:56:10.805813 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ac671b-a86e-4a73-a4ee-d6747df77289 tempest-VolumeQuotaClassesTest-526471283 tempest-VolumeQuotaClassesTest-526471283-project-admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:56:10.806088 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ac671b-a86e-4a73-a4ee-d6747df77289 tempest-VolumeQuotaClassesTest-526471283 tempest-VolumeQuotaClassesTest-526471283-project-admin] PATH_INFO: `/v3/projects` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:56:10.806728 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ac671b-a86e-4a73-a4ee-d6747df77289 tempest-VolumeQuotaClassesTest-526471283 tempest-VolumeQuotaClassesTest-526471283-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:56:10.809301 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ac671b-a86e-4a73-a4ee-d6747df77289 tempest-VolumeQuotaClassesTest-526471283 tempest-VolumeQuotaClassesTest-526471283-project-admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:56:10.917593 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1444/2878] 10.222.0.30 () {60 vars in 1153 bytes} [Tue Sep 6 04:56:10 2022] POST /identity/v3/projects => generated 353 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 04:56:10.922513 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea4578f-97f7-43f9-936c-dce7883857d2 tempest-VolumeQuotaClassesTest-526471283 tempest-VolumeQuotaClassesTest-526471283-project-admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:56:10.932383 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea4578f-97f7-43f9-936c-dce7883857d2 tempest-VolumeQuotaClassesTest-526471283 tempest-VolumeQuotaClassesTest-526471283-project-admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:56:10.989972 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11278ed-cfa9-4a79-9d24-cfdf30112f6f tempest-VolumeQuotaClassesTest-526471283 tempest-VolumeQuotaClassesTest-526471283-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b61e9860501422ca23a1fe1572cc53f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7f721731017347838f1f1e86dd8b8481', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:56:10.990524 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11278ed-cfa9-4a79-9d24-cfdf30112f6f tempest-VolumeQuotaClassesTest-526471283 tempest-VolumeQuotaClassesTest-526471283-project-admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:56:10.990791 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11278ed-cfa9-4a79-9d24-cfdf30112f6f tempest-VolumeQuotaClassesTest-526471283 tempest-VolumeQuotaClassesTest-526471283-project-admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:56:10.991068 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11278ed-cfa9-4a79-9d24-cfdf30112f6f tempest-VolumeQuotaClassesTest-526471283 tempest-VolumeQuotaClassesTest-526471283-project-admin] PATH_INFO: `/v3/domains` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:56:10.991713 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11278ed-cfa9-4a79-9d24-cfdf30112f6f tempest-VolumeQuotaClassesTest-526471283 tempest-VolumeQuotaClassesTest-526471283-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:56:10.992020 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11278ed-cfa9-4a79-9d24-cfdf30112f6f tempest-VolumeQuotaClassesTest-526471283 tempest-VolumeQuotaClassesTest-526471283-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:56:10.993646 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11278ed-cfa9-4a79-9d24-cfdf30112f6f tempest-VolumeQuotaClassesTest-526471283 tempest-VolumeQuotaClassesTest-526471283-project-admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:56:11.006842 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1435/2879] 10.222.0.30 () {58 vars in 1153 bytes} [Tue Sep 6 04:56:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:56:11.043801 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ac671b-a86e-4a73-a4ee-d6747df77289 tempest-VolumeQuotaClassesTest-526471283 tempest-VolumeQuotaClassesTest-526471283-project-admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:56:11.054618 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ac671b-a86e-4a73-a4ee-d6747df77289 tempest-VolumeQuotaClassesTest-526471283 tempest-VolumeQuotaClassesTest-526471283-project-admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:56:11.059143 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0b80dd-364c-47bb-beb6-fcae974a5044 tempest-VolumeQuotaClassesTest-526471283 tempest-VolumeQuotaClassesTest-526471283-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b61e9860501422ca23a1fe1572cc53f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7f721731017347838f1f1e86dd8b8481', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:56:11.059756 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0b80dd-364c-47bb-beb6-fcae974a5044 tempest-VolumeQuotaClassesTest-526471283 tempest-VolumeQuotaClassesTest-526471283-project-admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:56:11.060035 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0b80dd-364c-47bb-beb6-fcae974a5044 tempest-VolumeQuotaClassesTest-526471283 tempest-VolumeQuotaClassesTest-526471283-project-admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:56:11.060304 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0b80dd-364c-47bb-beb6-fcae974a5044 tempest-VolumeQuotaClassesTest-526471283 tempest-VolumeQuotaClassesTest-526471283-project-admin] PATH_INFO: `/v3/projects/e2aabf4f775e4aceb2e20718e10d75b9` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:56:11.061696 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0b80dd-364c-47bb-beb6-fcae974a5044 tempest-VolumeQuotaClassesTest-526471283 tempest-VolumeQuotaClassesTest-526471283-project-admin] RBAC: Authorizing `identity:delete_project(project_id=e2aabf4f775e4aceb2e20718e10d75b9)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:56:11.063574 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0b80dd-364c-47bb-beb6-fcae974a5044 tempest-VolumeQuotaClassesTest-526471283 tempest-VolumeQuotaClassesTest-526471283-project-admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:56:11.178488 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-fc0b80dd-364c-47bb-beb6-fcae974a5044 tempest-VolumeQuotaClassesTest-526471283 tempest-VolumeQuotaClassesTest-526471283-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e2aabf4f775e4aceb2e20718e10d75b9', 'request_id': 'req-fc0b80dd-364c-47bb-beb6-fcae974a5044'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:56:11.183914 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-fc0b80dd-364c-47bb-beb6-fcae974a5044 tempest-VolumeQuotaClassesTest-526471283 tempest-VolumeQuotaClassesTest-526471283-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e2aabf4f775e4aceb2e20718e10d75b9', 'request_id': 'req-fc0b80dd-364c-47bb-beb6-fcae974a5044'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:56:11.250905 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-fc0b80dd-364c-47bb-beb6-fcae974a5044 tempest-VolumeQuotaClassesTest-526471283 tempest-VolumeQuotaClassesTest-526471283-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e2aabf4f775e4aceb2e20718e10d75b9', 'request_id': 'req-fc0b80dd-364c-47bb-beb6-fcae974a5044'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:56:11.262724 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-fc0b80dd-364c-47bb-beb6-fcae974a5044 tempest-VolumeQuotaClassesTest-526471283 tempest-VolumeQuotaClassesTest-526471283-project-admin] The token cache is being invalidate because project e2aabf4f775e4aceb2e20718e10d75b9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81698) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 04:56:11.263015 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-fc0b80dd-364c-47bb-beb6-fcae974a5044 tempest-VolumeQuotaClassesTest-526471283 tempest-VolumeQuotaClassesTest-526471283-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:56:11.263781 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-fc0b80dd-364c-47bb-beb6-fcae974a5044 tempest-VolumeQuotaClassesTest-526471283 tempest-VolumeQuotaClassesTest-526471283-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:56:11.265269 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1445/2880] 10.222.0.30 () {58 vars in 1233 bytes} [Tue Sep 6 04:56:11 2022] DELETE /identity/v3/projects/e2aabf4f775e4aceb2e20718e10d75b9 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 04:56:11.880566 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11278ed-cfa9-4a79-9d24-cfdf30112f6f tempest-VolumeQuotaClassesTest-526471283 tempest-VolumeQuotaClassesTest-526471283-project-admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:56:11.884912 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-c11278ed-cfa9-4a79-9d24-cfdf30112f6f tempest-VolumeQuotaClassesTest-526471283 tempest-VolumeQuotaClassesTest-526471283-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:56:11.912872 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:56:11.947042 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11278ed-cfa9-4a79-9d24-cfdf30112f6f tempest-VolumeQuotaClassesTest-526471283 tempest-VolumeQuotaClassesTest-526471283-project-admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:56:12.003366 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680714af-a84a-4b3a-81ca-ccab77a46ade admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:56:12.003936 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680714af-a84a-4b3a-81ca-ccab77a46ade admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:56:12.004218 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680714af-a84a-4b3a-81ca-ccab77a46ade admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:56:12.004495 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680714af-a84a-4b3a-81ca-ccab77a46ade admin admin] PATH_INFO: `/v3/users/45555e9de7e44aa497ca47583673b1c5` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:56:12.005735 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680714af-a84a-4b3a-81ca-ccab77a46ade admin admin] RBAC: Authorizing `identity:delete_user(user_id=45555e9de7e44aa497ca47583673b1c5)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:56:12.007642 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680714af-a84a-4b3a-81ca-ccab77a46ade admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:56:12.219282 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-680714af-a84a-4b3a-81ca-ccab77a46ade admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '45555e9de7e44aa497ca47583673b1c5'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:56:12.285232 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-680714af-a84a-4b3a-81ca-ccab77a46ade admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '45555e9de7e44aa497ca47583673b1c5'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:56:12.285834 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-680714af-a84a-4b3a-81ca-ccab77a46ade admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '45555e9de7e44aa497ca47583673b1c5'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:56:12.293671 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-680714af-a84a-4b3a-81ca-ccab77a46ade admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '45555e9de7e44aa497ca47583673b1c5'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:56:12.305580 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-680714af-a84a-4b3a-81ca-ccab77a46ade admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '45555e9de7e44aa497ca47583673b1c5'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:56:12.311958 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1436/2881] 10.222.0.30 () {58 vars in 1224 bytes} [Tue Sep 6 04:56:11 2022] DELETE /identity/v3/users/45555e9de7e44aa497ca47583673b1c5 => generated 0 bytes in 432 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 04:56:12.316417 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a7b61c-640a-46d6-9569-913374ec0260 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:56:12.316900 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a7b61c-640a-46d6-9569-913374ec0260 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:56:12.317017 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a7b61c-640a-46d6-9569-913374ec0260 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:56:12.337857 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-82a7b61c-640a-46d6-9569-913374ec0260 None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:56:12.339064 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-82a7b61c-640a-46d6-9569-913374ec0260 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:56:12.387754 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:56:12.401266 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1446/2882] 10.222.0.30 () {58 vars in 958 bytes} [Tue Sep 6 04:56:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 04:56:12.411604 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680714af-a84a-4b3a-81ca-ccab77a46ade admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:56:12.413093 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-680714af-a84a-4b3a-81ca-ccab77a46ade admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:56:12.470806 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680714af-a84a-4b3a-81ca-ccab77a46ade admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:56:12.474433 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05375991-704d-44e8-ba47-541130f6bcac service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90492440f6a6471e8392361c098428aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b707733ce65c42edbc1fd95ad0a1c06c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:56:12.475035 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05375991-704d-44e8-ba47-541130f6bcac service neutron] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:56:12.475309 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05375991-704d-44e8-ba47-541130f6bcac service neutron] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:56:12.475578 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05375991-704d-44e8-ba47-541130f6bcac service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:56:12.482971 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:56:12.485646 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05375991-704d-44e8-ba47-541130f6bcac service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:56:12.487067 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05375991-704d-44e8-ba47-541130f6bcac service neutron] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:56:12.497558 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1437/2883] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Sep 6 04:56:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 04:56:12.887190 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a7b61c-640a-46d6-9569-913374ec0260 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:56:12.888650 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-82a7b61c-640a-46d6-9569-913374ec0260 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:56:12.901187 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a7b61c-640a-46d6-9569-913374ec0260 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:56:12.906200 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8687f72d-9aa6-4bb6-8e9f-192b808d3774 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:56:12.906839 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8687f72d-9aa6-4bb6-8e9f-192b808d3774 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:56:12.907122 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8687f72d-9aa6-4bb6-8e9f-192b808d3774 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:56:12.907395 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8687f72d-9aa6-4bb6-8e9f-192b808d3774 admin admin] PATH_INFO: `/v3/projects/08628313d57e46be90123e547d651d33` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:56:12.908527 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8687f72d-9aa6-4bb6-8e9f-192b808d3774 admin admin] RBAC: Authorizing `identity:delete_project(project_id=08628313d57e46be90123e547d651d33)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:56:12.910802 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8687f72d-9aa6-4bb6-8e9f-192b808d3774 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:56:13.010368 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-8687f72d-9aa6-4bb6-8e9f-192b808d3774 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '08628313d57e46be90123e547d651d33', 'request_id': 'req-8687f72d-9aa6-4bb6-8e9f-192b808d3774'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:56:13.016104 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-8687f72d-9aa6-4bb6-8e9f-192b808d3774 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '08628313d57e46be90123e547d651d33', 'request_id': 'req-8687f72d-9aa6-4bb6-8e9f-192b808d3774'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:56:13.098099 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-8687f72d-9aa6-4bb6-8e9f-192b808d3774 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '08628313d57e46be90123e547d651d33', 'request_id': 'req-8687f72d-9aa6-4bb6-8e9f-192b808d3774'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:56:13.109925 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-8687f72d-9aa6-4bb6-8e9f-192b808d3774 admin admin] The token cache is being invalidate because project 08628313d57e46be90123e547d651d33 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81698) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 04:56:13.113370 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-8687f72d-9aa6-4bb6-8e9f-192b808d3774 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:56:13.113370 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-8687f72d-9aa6-4bb6-8e9f-192b808d3774 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:56:13.113370 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1447/2884] 10.222.0.30 () {58 vars in 1233 bytes} [Tue Sep 6 04:56:12 2022] DELETE /identity/v3/projects/08628313d57e46be90123e547d651d33 => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 04:56:13.116459 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05375991-704d-44e8-ba47-541130f6bcac service neutron] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:56:13.126642 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05375991-704d-44e8-ba47-541130f6bcac service neutron] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:56:13.181862 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae224ab-3730-4328-aaca-f21e00acc14d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:56:13.182418 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae224ab-3730-4328-aaca-f21e00acc14d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:56:13.182694 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae224ab-3730-4328-aaca-f21e00acc14d admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:56:13.182963 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae224ab-3730-4328-aaca-f21e00acc14d admin admin] PATH_INFO: `/v3/users/7b61e9860501422ca23a1fe1572cc53f` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:56:13.184060 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae224ab-3730-4328-aaca-f21e00acc14d admin admin] RBAC: Authorizing `identity:delete_user(user_id=7b61e9860501422ca23a1fe1572cc53f)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:56:13.186036 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae224ab-3730-4328-aaca-f21e00acc14d admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:56:13.363767 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-7ae224ab-3730-4328-aaca-f21e00acc14d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7b61e9860501422ca23a1fe1572cc53f'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:56:13.446920 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-7ae224ab-3730-4328-aaca-f21e00acc14d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7b61e9860501422ca23a1fe1572cc53f'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:56:13.447476 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-7ae224ab-3730-4328-aaca-f21e00acc14d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7b61e9860501422ca23a1fe1572cc53f'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:56:13.455157 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-7ae224ab-3730-4328-aaca-f21e00acc14d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7b61e9860501422ca23a1fe1572cc53f'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:56:13.467310 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-7ae224ab-3730-4328-aaca-f21e00acc14d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7b61e9860501422ca23a1fe1572cc53f'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:56:13.473166 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1438/2885] 10.222.0.30 () {58 vars in 1224 bytes} [Tue Sep 6 04:56:13 2022] DELETE /identity/v3/users/7b61e9860501422ca23a1fe1572cc53f => generated 0 bytes in 357 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 04:56:13.900595 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8687f72d-9aa6-4bb6-8e9f-192b808d3774 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:56:13.903789 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-8687f72d-9aa6-4bb6-8e9f-192b808d3774 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:56:13.916115 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8687f72d-9aa6-4bb6-8e9f-192b808d3774 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:56:13.973080 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b479111f-deb6-4567-a8df-c2c2ca7b5152 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:56:13.973745 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b479111f-deb6-4567-a8df-c2c2ca7b5152 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:56:13.974018 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b479111f-deb6-4567-a8df-c2c2ca7b5152 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:56:13.974286 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b479111f-deb6-4567-a8df-c2c2ca7b5152 admin admin] PATH_INFO: `/v3/projects/7f721731017347838f1f1e86dd8b8481` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:56:13.975393 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b479111f-deb6-4567-a8df-c2c2ca7b5152 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7f721731017347838f1f1e86dd8b8481)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:56:13.977270 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b479111f-deb6-4567-a8df-c2c2ca7b5152 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:56:14.076273 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-b479111f-deb6-4567-a8df-c2c2ca7b5152 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7f721731017347838f1f1e86dd8b8481', 'request_id': 'req-b479111f-deb6-4567-a8df-c2c2ca7b5152'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:56:14.082405 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-b479111f-deb6-4567-a8df-c2c2ca7b5152 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7f721731017347838f1f1e86dd8b8481', 'request_id': 'req-b479111f-deb6-4567-a8df-c2c2ca7b5152'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:56:14.161653 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-b479111f-deb6-4567-a8df-c2c2ca7b5152 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7f721731017347838f1f1e86dd8b8481', 'request_id': 'req-b479111f-deb6-4567-a8df-c2c2ca7b5152'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:56:14.173848 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-b479111f-deb6-4567-a8df-c2c2ca7b5152 admin admin] The token cache is being invalidate because project 7f721731017347838f1f1e86dd8b8481 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81698) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 04:56:14.174134 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-b479111f-deb6-4567-a8df-c2c2ca7b5152 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:56:14.174747 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-b479111f-deb6-4567-a8df-c2c2ca7b5152 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:56:14.176208 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1448/2886] 10.222.0.30 () {58 vars in 1233 bytes} [Tue Sep 6 04:56:13 2022] DELETE /identity/v3/projects/7f721731017347838f1f1e86dd8b8481 => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 04:56:14.181243 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a4d5df-eb5e-4e8b-bfc9-9d8e248ff92a None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:56:14.181772 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a4d5df-eb5e-4e8b-bfc9-9d8e248ff92a None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:56:14.181881 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a4d5df-eb5e-4e8b-bfc9-9d8e248ff92a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:56:14.202153 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-b5a4d5df-eb5e-4e8b-bfc9-9d8e248ff92a None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:56:14.203345 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-b5a4d5df-eb5e-4e8b-bfc9-9d8e248ff92a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:56:14.261525 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1439/2887] 10.222.0.30 () {58 vars in 958 bytes} [Tue Sep 6 04:56:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 04:56:14.270315 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4060b65f-f690-4465-9899-c9f37249ef7d None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:56:14.270601 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4060b65f-f690-4465-9899-c9f37249ef7d None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:56:14.270874 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4060b65f-f690-4465-9899-c9f37249ef7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:56:14.285278 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:56:14.290747 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-4060b65f-f690-4465-9899-c9f37249ef7d None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:56:14.297453 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-4060b65f-f690-4465-9899-c9f37249ef7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:56:14.300850 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1449/2888] 10.222.0.30 () {58 vars in 958 bytes} [Tue Sep 6 04:56:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 04:56:14.305119 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a4d5df-eb5e-4e8b-bfc9-9d8e248ff92a None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:56:14.315962 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a4d5df-eb5e-4e8b-bfc9-9d8e248ff92a None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:56:14.320560 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf29c40-8f42-4887-8415-65e700de4473 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:56:14.326358 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf29c40-8f42-4887-8415-65e700de4473 admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:56:14.326358 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf29c40-8f42-4887-8415-65e700de4473 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:56:14.326358 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf29c40-8f42-4887-8415-65e700de4473 admin admin] PATH_INFO: `/v3/domains` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:56:14.326358 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf29c40-8f42-4887-8415-65e700de4473 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:56:14.326358 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf29c40-8f42-4887-8415-65e700de4473 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:56:14.326358 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf29c40-8f42-4887-8415-65e700de4473 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:56:14.336290 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1440/2889] 10.222.0.30 () {58 vars in 1153 bytes} [Tue Sep 6 04:56:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:56:14.340590 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4060b65f-f690-4465-9899-c9f37249ef7d None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:56:14.350037 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4060b65f-f690-4465-9899-c9f37249ef7d None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:56:14.354872 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a350d113-39fc-4d6d-ae82-3c5004017824 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:56:14.355480 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a350d113-39fc-4d6d-ae82-3c5004017824 admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:56:14.355978 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a350d113-39fc-4d6d-ae82-3c5004017824 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:56:14.356468 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a350d113-39fc-4d6d-ae82-3c5004017824 admin admin] PATH_INFO: `/v3/projects` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:56:14.357139 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a350d113-39fc-4d6d-ae82-3c5004017824 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:56:14.359642 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a350d113-39fc-4d6d-ae82-3c5004017824 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:56:14.444465 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1450/2890] 10.222.0.30 () {60 vars in 1153 bytes} [Tue Sep 6 04:56:14 2022] POST /identity/v3/projects => generated 367 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 04:56:14.448869 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf29c40-8f42-4887-8415-65e700de4473 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:56:14.454858 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:56:14.460219 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf29c40-8f42-4887-8415-65e700de4473 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:56:14.515620 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec2388d-b931-4099-9c9f-5dc5959c24e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:56:14.516331 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec2388d-b931-4099-9c9f-5dc5959c24e9 admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:56:14.516636 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec2388d-b931-4099-9c9f-5dc5959c24e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:56:14.517046 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec2388d-b931-4099-9c9f-5dc5959c24e9 admin admin] PATH_INFO: `/v3/users` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:56:14.518190 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec2388d-b931-4099-9c9f-5dc5959c24e9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:56:14.520250 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec2388d-b931-4099-9c9f-5dc5959c24e9 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:56:14.625436 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1441/2891] 10.222.0.30 () {60 vars in 1144 bytes} [Tue Sep 6 04:56:14 2022] POST /identity/v3/users => generated 284 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 04:56:14.629408 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a350d113-39fc-4d6d-ae82-3c5004017824 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:56:14.640420 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a350d113-39fc-4d6d-ae82-3c5004017824 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:56:14.644946 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e81328f-d891-4d85-af0b-fc7db5ebfd71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:56:14.645625 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e81328f-d891-4d85-af0b-fc7db5ebfd71 admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:56:14.645901 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e81328f-d891-4d85-af0b-fc7db5ebfd71 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:56:14.646166 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e81328f-d891-4d85-af0b-fc7db5ebfd71 admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:56:14.646800 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e81328f-d891-4d85-af0b-fc7db5ebfd71 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:56:14.648377 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e81328f-d891-4d85-af0b-fc7db5ebfd71 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:56:14.658173 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1451/2892] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Sep 6 04:56:14 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 04:56:14.662307 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec2388d-b931-4099-9c9f-5dc5959c24e9 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:56:14.672172 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec2388d-b931-4099-9c9f-5dc5959c24e9 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:56:14.676768 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d474e3b0-9369-4268-a10f-eb6b4f597b18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:56:14.677405 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d474e3b0-9369-4268-a10f-eb6b4f597b18 admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:56:14.677691 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d474e3b0-9369-4268-a10f-eb6b4f597b18 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:56:14.677968 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d474e3b0-9369-4268-a10f-eb6b4f597b18 admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:56:14.678607 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d474e3b0-9369-4268-a10f-eb6b4f597b18 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:56:14.680224 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d474e3b0-9369-4268-a10f-eb6b4f597b18 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:56:14.691109 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1442/2893] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Sep 6 04:56:14 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 04:56:14.694996 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e81328f-d891-4d85-af0b-fc7db5ebfd71 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:56:14.705327 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e81328f-d891-4d85-af0b-fc7db5ebfd71 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:56:14.709853 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd089f35-ff33-412d-8678-7137625c6174 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:56:14.710486 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd089f35-ff33-412d-8678-7137625c6174 admin admin] REQUEST_METHOD: `PUT` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:56:14.710764 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd089f35-ff33-412d-8678-7137625c6174 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:56:14.711035 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd089f35-ff33-412d-8678-7137625c6174 admin admin] PATH_INFO: `/v3/projects/80e4ff336b3c47c5b1ab398a98adc4bb/users/c2ac9f78002f41c0b17b4f9e3ac560e8/roles/29607b2de39f40d68bb6ceef64b8a732` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:56:14.725566 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd089f35-ff33-412d-8678-7137625c6174 admin admin] RBAC: Authorizing `identity:create_grant(project_id=80e4ff336b3c47c5b1ab398a98adc4bb, user_id=c2ac9f78002f41c0b17b4f9e3ac560e8, role_id=29607b2de39f40d68bb6ceef64b8a732)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:56:14.728207 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd089f35-ff33-412d-8678-7137625c6174 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:56:14.830901 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1452/2894] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Sep 6 04:56:14 2022] PUT /identity/v3/projects/80e4ff336b3c47c5b1ab398a98adc4bb/users/c2ac9f78002f41c0b17b4f9e3ac560e8/roles/29607b2de39f40d68bb6ceef64b8a732 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 04:56:14.841425 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d917062-a7ed-410c-bfc8-50e804abd1eb None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:56:14.841777 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d917062-a7ed-410c-bfc8-50e804abd1eb None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:56:14.842189 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d917062-a7ed-410c-bfc8-50e804abd1eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:56:14.870324 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-6d917062-a7ed-410c-bfc8-50e804abd1eb None None] MFA Rules not processed for user `c2ac9f78002f41c0b17b4f9e3ac560e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:56:14.871578 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-6d917062-a7ed-410c-bfc8-50e804abd1eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:56:14.938440 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1443/2895] 10.222.0.30 () {58 vars in 958 bytes} [Tue Sep 6 04:56:14 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 04:56:14.942981 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd089f35-ff33-412d-8678-7137625c6174 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:56:15.005409 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd089f35-ff33-412d-8678-7137625c6174 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:56:15.010305 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84b4cea-7292-4b3b-aefe-dbc7cfd0dcee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:56:15.010913 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84b4cea-7292-4b3b-aefe-dbc7cfd0dcee admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:56:15.011193 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84b4cea-7292-4b3b-aefe-dbc7cfd0dcee admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:56:15.011466 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84b4cea-7292-4b3b-aefe-dbc7cfd0dcee admin admin] PATH_INFO: `/v3/projects` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:56:15.012135 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b84b4cea-7292-4b3b-aefe-dbc7cfd0dcee admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:56:15.014089 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b84b4cea-7292-4b3b-aefe-dbc7cfd0dcee admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:56:15.093531 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1453/2896] 10.222.0.30 () {60 vars in 1153 bytes} [Tue Sep 6 04:56:14 2022] POST /identity/v3/projects => generated 367 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 04:56:15.097486 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d917062-a7ed-410c-bfc8-50e804abd1eb None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:56:15.109511 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d917062-a7ed-410c-bfc8-50e804abd1eb None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:56:15.164243 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b185dca-5c6e-4f00-8c86-7fe31f1a3e89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:56:15.164845 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b185dca-5c6e-4f00-8c86-7fe31f1a3e89 admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:56:15.165153 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b185dca-5c6e-4f00-8c86-7fe31f1a3e89 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:56:15.165458 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b185dca-5c6e-4f00-8c86-7fe31f1a3e89 admin admin] PATH_INFO: `/v3/users` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:56:15.166073 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b185dca-5c6e-4f00-8c86-7fe31f1a3e89 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:56:15.167947 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b185dca-5c6e-4f00-8c86-7fe31f1a3e89 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:56:15.236673 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:56:15.241641 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1444/2897] 10.222.0.30 () {60 vars in 1144 bytes} [Tue Sep 6 04:56:15 2022] POST /identity/v3/users => generated 290 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 04:56:15.245659 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84b4cea-7292-4b3b-aefe-dbc7cfd0dcee admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:56:15.259661 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84b4cea-7292-4b3b-aefe-dbc7cfd0dcee admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:56:15.264993 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e916c81-2efd-4390-b7ee-64adfa9cbb9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:56:15.265645 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e916c81-2efd-4390-b7ee-64adfa9cbb9d admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:56:15.265930 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e916c81-2efd-4390-b7ee-64adfa9cbb9d admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:56:15.266204 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e916c81-2efd-4390-b7ee-64adfa9cbb9d admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:56:15.267329 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e916c81-2efd-4390-b7ee-64adfa9cbb9d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:56:15.268991 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e916c81-2efd-4390-b7ee-64adfa9cbb9d admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:56:15.279947 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1454/2898] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Sep 6 04:56:15 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 04:56:15.285170 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b185dca-5c6e-4f00-8c86-7fe31f1a3e89 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:56:15.295054 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b185dca-5c6e-4f00-8c86-7fe31f1a3e89 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:56:15.300877 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4abd8aa-d201-47af-9d95-e5ed376ab05b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:56:15.301592 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4abd8aa-d201-47af-9d95-e5ed376ab05b admin admin] REQUEST_METHOD: `PUT` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:56:15.301870 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4abd8aa-d201-47af-9d95-e5ed376ab05b admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:56:15.302147 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4abd8aa-d201-47af-9d95-e5ed376ab05b admin admin] PATH_INFO: `/v3/projects/71be7909bb7e439b8dcd08cd64670a6f/users/2548634937cf496ba2a517880c33c920/roles/ea4c814713b149108e488de855b025e4` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:56:15.316733 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4abd8aa-d201-47af-9d95-e5ed376ab05b admin admin] RBAC: Authorizing `identity:create_grant(project_id=71be7909bb7e439b8dcd08cd64670a6f, user_id=2548634937cf496ba2a517880c33c920, role_id=ea4c814713b149108e488de855b025e4)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:56:15.319371 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4abd8aa-d201-47af-9d95-e5ed376ab05b admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:56:15.388342 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1445/2899] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Sep 6 04:56:15 2022] PUT /identity/v3/projects/71be7909bb7e439b8dcd08cd64670a6f/users/2548634937cf496ba2a517880c33c920/roles/ea4c814713b149108e488de855b025e4 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 04:56:15.392897 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e916c81-2efd-4390-b7ee-64adfa9cbb9d admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:56:15.403104 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e916c81-2efd-4390-b7ee-64adfa9cbb9d admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:56:15.461265 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd4a0e1-1997-4653-acfe-370414e6a462 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:56:15.461900 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd4a0e1-1997-4653-acfe-370414e6a462 admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:56:15.462185 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd4a0e1-1997-4653-acfe-370414e6a462 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:56:15.462455 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd4a0e1-1997-4653-acfe-370414e6a462 admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:56:15.463090 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd4a0e1-1997-4653-acfe-370414e6a462 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:56:15.464679 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd4a0e1-1997-4653-acfe-370414e6a462 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:56:15.474785 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1455/2900] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Sep 6 04:56:15 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 04:56:15.479001 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4abd8aa-d201-47af-9d95-e5ed376ab05b admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:56:15.490538 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4abd8aa-d201-47af-9d95-e5ed376ab05b admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:56:15.495169 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3d5279-6fae-496c-9628-3f65ab9e2723 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:56:15.495837 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3d5279-6fae-496c-9628-3f65ab9e2723 admin admin] REQUEST_METHOD: `PUT` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:56:15.496111 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3d5279-6fae-496c-9628-3f65ab9e2723 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:56:15.496378 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3d5279-6fae-496c-9628-3f65ab9e2723 admin admin] PATH_INFO: `/v3/projects/71be7909bb7e439b8dcd08cd64670a6f/users/2548634937cf496ba2a517880c33c920/roles/ea4c814713b149108e488de855b025e4` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:56:15.498311 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3d5279-6fae-496c-9628-3f65ab9e2723 admin admin] RBAC: Authorizing `identity:create_grant(project_id=71be7909bb7e439b8dcd08cd64670a6f, user_id=2548634937cf496ba2a517880c33c920, role_id=ea4c814713b149108e488de855b025e4)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:56:15.500893 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3d5279-6fae-496c-9628-3f65ab9e2723 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:56:15.511095 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1446/2901] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Sep 6 04:56:15 2022] PUT /identity/v3/projects/71be7909bb7e439b8dcd08cd64670a6f/users/2548634937cf496ba2a517880c33c920/roles/ea4c814713b149108e488de855b025e4 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 04:56:15.521964 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589f1ff1-b8f5-4f2b-aaa6-a03162b6604b None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:56:15.522258 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589f1ff1-b8f5-4f2b-aaa6-a03162b6604b None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:56:15.522613 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589f1ff1-b8f5-4f2b-aaa6-a03162b6604b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:56:15.549057 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:56:15.552111 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-589f1ff1-b8f5-4f2b-aaa6-a03162b6604b None None] MFA Rules not processed for user `2548634937cf496ba2a517880c33c920`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:56:15.553355 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-589f1ff1-b8f5-4f2b-aaa6-a03162b6604b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:56:15.622056 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1456/2902] 10.222.0.30 () {58 vars in 958 bytes} [Tue Sep 6 04:56:15 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 04:56:15.633665 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3d5279-6fae-496c-9628-3f65ab9e2723 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:56:15.635427 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-5c3d5279-6fae-496c-9628-3f65ab9e2723 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:56:15.693909 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3d5279-6fae-496c-9628-3f65ab9e2723 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:56:15.697544 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d28df1-4a39-4fb1-853b-47fccedbf078 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd09b5ab675634a2eb4b03755743f7d0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b707733ce65c42edbc1fd95ad0a1c06c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:56:15.698142 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d28df1-4a39-4fb1-853b-47fccedbf078 service cinder] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:56:15.698420 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d28df1-4a39-4fb1-853b-47fccedbf078 service cinder] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:56:15.698692 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d28df1-4a39-4fb1-853b-47fccedbf078 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:56:15.708251 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d28df1-4a39-4fb1-853b-47fccedbf078 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:56:15.709719 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d28df1-4a39-4fb1-853b-47fccedbf078 service cinder] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:56:15.719552 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1447/2903] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Sep 6 04:56:15 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 04:56:15.944080 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589f1ff1-b8f5-4f2b-aaa6-a03162b6604b None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:56:15.955171 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589f1ff1-b8f5-4f2b-aaa6-a03162b6604b None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:56:15.959382 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e557dd9-5cd2-421e-bca2-d158f71a15e0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd09b5ab675634a2eb4b03755743f7d0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b707733ce65c42edbc1fd95ad0a1c06c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:56:15.966945 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e557dd9-5cd2-421e-bca2-d158f71a15e0 service cinder] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:56:15.966945 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e557dd9-5cd2-421e-bca2-d158f71a15e0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:56:15.966945 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e557dd9-5cd2-421e-bca2-d158f71a15e0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:56:16.026692 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e557dd9-5cd2-421e-bca2-d158f71a15e0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:56:16.028314 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e557dd9-5cd2-421e-bca2-d158f71a15e0 service cinder] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:56:16.038756 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1457/2904] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Sep 6 04:56:15 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 04:56:22.359406 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d28df1-4a39-4fb1-853b-47fccedbf078 service cinder] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:56:22.422529 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d28df1-4a39-4fb1-853b-47fccedbf078 service cinder] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:56:22.427042 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4241df6-f189-447d-a8a9-9a719f8792ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:56:22.427664 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4241df6-f189-447d-a8a9-9a719f8792ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:56:22.427940 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4241df6-f189-447d-a8a9-9a719f8792ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:56:22.428210 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4241df6-f189-447d-a8a9-9a719f8792ca admin admin] PATH_INFO: `/v3/users/c2ac9f78002f41c0b17b4f9e3ac560e8` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:56:22.429327 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4241df6-f189-447d-a8a9-9a719f8792ca admin admin] RBAC: Authorizing `identity:delete_user(user_id=c2ac9f78002f41c0b17b4f9e3ac560e8)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:56:22.431230 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4241df6-f189-447d-a8a9-9a719f8792ca admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:56:22.563255 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-a4241df6-f189-447d-a8a9-9a719f8792ca admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c2ac9f78002f41c0b17b4f9e3ac560e8'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:56:22.606822 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-a4241df6-f189-447d-a8a9-9a719f8792ca admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c2ac9f78002f41c0b17b4f9e3ac560e8'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:56:22.607434 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-a4241df6-f189-447d-a8a9-9a719f8792ca admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c2ac9f78002f41c0b17b4f9e3ac560e8'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:56:22.614513 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-a4241df6-f189-447d-a8a9-9a719f8792ca admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c2ac9f78002f41c0b17b4f9e3ac560e8'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:56:22.618947 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=89,keystone:INSERT=3,keystone:DELETE=8 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:56:22.627748 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-a4241df6-f189-447d-a8a9-9a719f8792ca admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c2ac9f78002f41c0b17b4f9e3ac560e8'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:56:22.634297 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1448/2905] 10.222.0.30 () {58 vars in 1224 bytes} [Tue Sep 6 04:56:22 2022] DELETE /identity/v3/users/c2ac9f78002f41c0b17b4f9e3ac560e8 => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 04:56:22.639122 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5762f932-f9e7-4a76-92fb-3df118b5eb06 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:56:22.640201 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5762f932-f9e7-4a76-92fb-3df118b5eb06 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:56:22.640201 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5762f932-f9e7-4a76-92fb-3df118b5eb06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:56:22.661625 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-5762f932-f9e7-4a76-92fb-3df118b5eb06 None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:56:22.663010 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-5762f932-f9e7-4a76-92fb-3df118b5eb06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:56:22.729234 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1458/2906] 10.222.0.30 () {58 vars in 958 bytes} [Tue Sep 6 04:56:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 04:56:22.739405 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4241df6-f189-447d-a8a9-9a719f8792ca admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:56:22.740938 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-a4241df6-f189-447d-a8a9-9a719f8792ca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:56:22.799987 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4241df6-f189-447d-a8a9-9a719f8792ca admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:56:22.803627 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0bfe414-394a-4577-8a65-6367f03962af service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90492440f6a6471e8392361c098428aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b707733ce65c42edbc1fd95ad0a1c06c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:56:22.804237 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bfe414-394a-4577-8a65-6367f03962af service neutron] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:56:22.804514 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bfe414-394a-4577-8a65-6367f03962af service neutron] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:56:22.804785 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bfe414-394a-4577-8a65-6367f03962af service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:56:22.814666 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0bfe414-394a-4577-8a65-6367f03962af service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:56:22.816029 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0bfe414-394a-4577-8a65-6367f03962af service neutron] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:56:22.825536 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1449/2907] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Sep 6 04:56:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 04:56:23.220386 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5762f932-f9e7-4a76-92fb-3df118b5eb06 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:56:23.224602 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-5762f932-f9e7-4a76-92fb-3df118b5eb06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:56:23.235036 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5762f932-f9e7-4a76-92fb-3df118b5eb06 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:56:23.239689 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34f0fa6-b7cb-4ef2-8974-162d95a26185 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:56:23.242193 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34f0fa6-b7cb-4ef2-8974-162d95a26185 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:56:23.242193 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34f0fa6-b7cb-4ef2-8974-162d95a26185 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:56:23.242193 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34f0fa6-b7cb-4ef2-8974-162d95a26185 admin admin] PATH_INFO: `/v3/projects/80e4ff336b3c47c5b1ab398a98adc4bb` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:56:23.242193 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34f0fa6-b7cb-4ef2-8974-162d95a26185 admin admin] RBAC: Authorizing `identity:delete_project(project_id=80e4ff336b3c47c5b1ab398a98adc4bb)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:56:23.243967 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34f0fa6-b7cb-4ef2-8974-162d95a26185 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:56:23.334198 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:56:23.346174 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-b34f0fa6-b7cb-4ef2-8974-162d95a26185 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '80e4ff336b3c47c5b1ab398a98adc4bb', 'request_id': 'req-b34f0fa6-b7cb-4ef2-8974-162d95a26185'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:56:23.351454 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-b34f0fa6-b7cb-4ef2-8974-162d95a26185 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '80e4ff336b3c47c5b1ab398a98adc4bb', 'request_id': 'req-b34f0fa6-b7cb-4ef2-8974-162d95a26185'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:56:23.424311 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-b34f0fa6-b7cb-4ef2-8974-162d95a26185 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '80e4ff336b3c47c5b1ab398a98adc4bb', 'request_id': 'req-b34f0fa6-b7cb-4ef2-8974-162d95a26185'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:56:23.435690 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-b34f0fa6-b7cb-4ef2-8974-162d95a26185 admin admin] The token cache is being invalidate because project 80e4ff336b3c47c5b1ab398a98adc4bb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81698) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 04:56:23.435980 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-b34f0fa6-b7cb-4ef2-8974-162d95a26185 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:56:23.437506 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-b34f0fa6-b7cb-4ef2-8974-162d95a26185 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:56:23.438983 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1459/2908] 10.222.0.30 () {58 vars in 1233 bytes} [Tue Sep 6 04:56:23 2022] DELETE /identity/v3/projects/80e4ff336b3c47c5b1ab398a98adc4bb => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 04:56:23.442992 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0bfe414-394a-4577-8a65-6367f03962af service neutron] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:56:23.453286 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0bfe414-394a-4577-8a65-6367f03962af service neutron] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:56:23.510219 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c20ed2-b037-4f1e-b3f6-40a91ca73343 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:56:23.510847 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c20ed2-b037-4f1e-b3f6-40a91ca73343 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:56:23.511129 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c20ed2-b037-4f1e-b3f6-40a91ca73343 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:56:23.511402 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c20ed2-b037-4f1e-b3f6-40a91ca73343 admin admin] PATH_INFO: `/v3/users/2548634937cf496ba2a517880c33c920` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:56:23.512518 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c20ed2-b037-4f1e-b3f6-40a91ca73343 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2548634937cf496ba2a517880c33c920)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:56:23.515051 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c20ed2-b037-4f1e-b3f6-40a91ca73343 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:56:23.699036 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-d6c20ed2-b037-4f1e-b3f6-40a91ca73343 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2548634937cf496ba2a517880c33c920'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:56:23.761329 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-d6c20ed2-b037-4f1e-b3f6-40a91ca73343 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2548634937cf496ba2a517880c33c920'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:56:23.761965 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-d6c20ed2-b037-4f1e-b3f6-40a91ca73343 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2548634937cf496ba2a517880c33c920'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:56:23.769795 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-d6c20ed2-b037-4f1e-b3f6-40a91ca73343 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2548634937cf496ba2a517880c33c920'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:56:23.781377 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-d6c20ed2-b037-4f1e-b3f6-40a91ca73343 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2548634937cf496ba2a517880c33c920'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:56:23.787402 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1450/2909] 10.222.0.30 () {58 vars in 1224 bytes} [Tue Sep 6 04:56:23 2022] DELETE /identity/v3/users/2548634937cf496ba2a517880c33c920 => generated 0 bytes in 345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 04:56:24.227181 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34f0fa6-b7cb-4ef2-8974-162d95a26185 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:56:24.229162 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-b34f0fa6-b7cb-4ef2-8974-162d95a26185 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:56:24.249417 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34f0fa6-b7cb-4ef2-8974-162d95a26185 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:56:24.307538 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0938a21-0150-4c2c-b00f-11e430e29f46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:56:24.308172 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0938a21-0150-4c2c-b00f-11e430e29f46 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:56:24.308453 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0938a21-0150-4c2c-b00f-11e430e29f46 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:56:24.308727 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0938a21-0150-4c2c-b00f-11e430e29f46 admin admin] PATH_INFO: `/v3/projects/71be7909bb7e439b8dcd08cd64670a6f` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:56:24.309890 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0938a21-0150-4c2c-b00f-11e430e29f46 admin admin] RBAC: Authorizing `identity:delete_project(project_id=71be7909bb7e439b8dcd08cd64670a6f)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:56:24.312833 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0938a21-0150-4c2c-b00f-11e430e29f46 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:56:24.428714 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-b0938a21-0150-4c2c-b00f-11e430e29f46 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '71be7909bb7e439b8dcd08cd64670a6f', 'request_id': 'req-b0938a21-0150-4c2c-b00f-11e430e29f46'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:56:24.434586 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-b0938a21-0150-4c2c-b00f-11e430e29f46 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '71be7909bb7e439b8dcd08cd64670a6f', 'request_id': 'req-b0938a21-0150-4c2c-b00f-11e430e29f46'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:56:24.502227 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-b0938a21-0150-4c2c-b00f-11e430e29f46 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '71be7909bb7e439b8dcd08cd64670a6f', 'request_id': 'req-b0938a21-0150-4c2c-b00f-11e430e29f46'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:56:24.514567 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-b0938a21-0150-4c2c-b00f-11e430e29f46 admin admin] The token cache is being invalidate because project 71be7909bb7e439b8dcd08cd64670a6f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81698) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 04:56:24.514900 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-b0938a21-0150-4c2c-b00f-11e430e29f46 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:56:24.516267 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-b0938a21-0150-4c2c-b00f-11e430e29f46 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:56:24.518037 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1460/2910] 10.222.0.30 () {58 vars in 1233 bytes} [Tue Sep 6 04:56:24 2022] DELETE /identity/v3/projects/71be7909bb7e439b8dcd08cd64670a6f => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 04:56:24.522666 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df88faac-f0e9-4794-8fe6-adc83fa7a6df None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:56:24.523215 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df88faac-f0e9-4794-8fe6-adc83fa7a6df None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:56:24.523313 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df88faac-f0e9-4794-8fe6-adc83fa7a6df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:56:24.538292 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:DELETE=12,keystone:SELECT=87,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:56:24.544741 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-df88faac-f0e9-4794-8fe6-adc83fa7a6df None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:56:24.546131 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-df88faac-f0e9-4794-8fe6-adc83fa7a6df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:56:24.610818 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1451/2911] 10.222.0.30 () {58 vars in 958 bytes} [Tue Sep 6 04:56:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 04:56:24.618871 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2401d02d-4f1d-4664-9dc6-7b0abc4ddc19 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:56:24.619271 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2401d02d-4f1d-4664-9dc6-7b0abc4ddc19 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:56:24.619649 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2401d02d-4f1d-4664-9dc6-7b0abc4ddc19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:56:24.641463 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-2401d02d-4f1d-4664-9dc6-7b0abc4ddc19 None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:56:24.642683 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-2401d02d-4f1d-4664-9dc6-7b0abc4ddc19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:56:24.650628 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1461/2912] 10.222.0.30 () {58 vars in 958 bytes} [Tue Sep 6 04:56:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 04:56:24.654946 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df88faac-f0e9-4794-8fe6-adc83fa7a6df None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:56:24.665232 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df88faac-f0e9-4794-8fe6-adc83fa7a6df None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:56:24.670374 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ad0f54-4c70-4c7e-96e6-908d73de0c44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:56:24.671704 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ad0f54-4c70-4c7e-96e6-908d73de0c44 admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:56:24.672059 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ad0f54-4c70-4c7e-96e6-908d73de0c44 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:56:24.672410 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ad0f54-4c70-4c7e-96e6-908d73de0c44 admin admin] PATH_INFO: `/v3/domains` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:56:24.673142 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ad0f54-4c70-4c7e-96e6-908d73de0c44 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:56:24.673555 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ad0f54-4c70-4c7e-96e6-908d73de0c44 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:56:24.675172 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ad0f54-4c70-4c7e-96e6-908d73de0c44 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:56:24.688247 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1452/2913] 10.222.0.30 () {58 vars in 1153 bytes} [Tue Sep 6 04:56:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 04:56:24.692573 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2401d02d-4f1d-4664-9dc6-7b0abc4ddc19 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:56:24.702002 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2401d02d-4f1d-4664-9dc6-7b0abc4ddc19 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:56:24.706660 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2283f1d-82af-4c72-a802-22a84fe8cc73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:56:24.707257 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2283f1d-82af-4c72-a802-22a84fe8cc73 admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:56:24.707528 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2283f1d-82af-4c72-a802-22a84fe8cc73 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:56:24.707794 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2283f1d-82af-4c72-a802-22a84fe8cc73 admin admin] PATH_INFO: `/v3/projects` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:56:24.708426 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2283f1d-82af-4c72-a802-22a84fe8cc73 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:56:24.710356 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2283f1d-82af-4c72-a802-22a84fe8cc73 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:56:24.813038 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1462/2914] 10.222.0.30 () {60 vars in 1153 bytes} [Tue Sep 6 04:56:24 2022] POST /identity/v3/projects => generated 371 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 04:56:24.817189 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ad0f54-4c70-4c7e-96e6-908d73de0c44 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:56:24.827409 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ad0f54-4c70-4c7e-96e6-908d73de0c44 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:56:24.885484 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17323875-7105-431a-8e49-1530e61f441c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:56:24.886095 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17323875-7105-431a-8e49-1530e61f441c admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:56:24.886372 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17323875-7105-431a-8e49-1530e61f441c admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:56:24.886643 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17323875-7105-431a-8e49-1530e61f441c admin admin] PATH_INFO: `/v3/users` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:56:24.887277 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17323875-7105-431a-8e49-1530e61f441c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:56:24.889328 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17323875-7105-431a-8e49-1530e61f441c admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:56:24.973106 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1453/2915] 10.222.0.30 () {60 vars in 1144 bytes} [Tue Sep 6 04:56:24 2022] POST /identity/v3/users => generated 286 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 04:56:24.977352 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2283f1d-82af-4c72-a802-22a84fe8cc73 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:56:24.989190 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2283f1d-82af-4c72-a802-22a84fe8cc73 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:56:24.993860 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1de02c-cc89-4a9a-b0db-f1273aeb208b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:56:24.994466 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1de02c-cc89-4a9a-b0db-f1273aeb208b admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:56:24.994746 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1de02c-cc89-4a9a-b0db-f1273aeb208b admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:56:24.995016 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1de02c-cc89-4a9a-b0db-f1273aeb208b admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:56:24.995661 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1de02c-cc89-4a9a-b0db-f1273aeb208b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:56:24.997254 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1de02c-cc89-4a9a-b0db-f1273aeb208b admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:56:25.007420 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1463/2916] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Sep 6 04:56:24 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 04:56:25.011458 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17323875-7105-431a-8e49-1530e61f441c admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:56:25.021758 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17323875-7105-431a-8e49-1530e61f441c admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:56:25.026360 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84172390-10ad-42f0-88ef-78d59ff8574d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:56:25.026972 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84172390-10ad-42f0-88ef-78d59ff8574d admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:56:25.027255 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84172390-10ad-42f0-88ef-78d59ff8574d admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:56:25.027530 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84172390-10ad-42f0-88ef-78d59ff8574d admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:56:25.028173 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84172390-10ad-42f0-88ef-78d59ff8574d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:56:25.030314 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84172390-10ad-42f0-88ef-78d59ff8574d admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:56:25.041084 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1454/2917] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Sep 6 04:56:25 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 04:56:25.045176 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1de02c-cc89-4a9a-b0db-f1273aeb208b admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:56:25.055708 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1de02c-cc89-4a9a-b0db-f1273aeb208b admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:56:25.060312 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025ca9e6-f605-4c59-b3ed-af4afbf2bd88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:56:25.060952 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025ca9e6-f605-4c59-b3ed-af4afbf2bd88 admin admin] REQUEST_METHOD: `PUT` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:56:25.061277 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025ca9e6-f605-4c59-b3ed-af4afbf2bd88 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:56:25.061579 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025ca9e6-f605-4c59-b3ed-af4afbf2bd88 admin admin] PATH_INFO: `/v3/projects/f23ca2e000164b9bada448dc01321bf8/users/fbf2a82ddde7434c85a6ca84ea332026/roles/29607b2de39f40d68bb6ceef64b8a732` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:56:25.069967 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:56:25.077436 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025ca9e6-f605-4c59-b3ed-af4afbf2bd88 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f23ca2e000164b9bada448dc01321bf8, user_id=fbf2a82ddde7434c85a6ca84ea332026, role_id=29607b2de39f40d68bb6ceef64b8a732)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:56:25.084145 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025ca9e6-f605-4c59-b3ed-af4afbf2bd88 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:56:25.160435 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1464/2918] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Sep 6 04:56:25 2022] PUT /identity/v3/projects/f23ca2e000164b9bada448dc01321bf8/users/fbf2a82ddde7434c85a6ca84ea332026/roles/29607b2de39f40d68bb6ceef64b8a732 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 04:56:25.170846 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea88617e-4218-438e-85d6-c7db3735da27 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:56:25.171160 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea88617e-4218-438e-85d6-c7db3735da27 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:56:25.171432 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea88617e-4218-438e-85d6-c7db3735da27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:56:25.199780 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-ea88617e-4218-438e-85d6-c7db3735da27 None None] MFA Rules not processed for user `fbf2a82ddde7434c85a6ca84ea332026`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:56:25.201013 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-ea88617e-4218-438e-85d6-c7db3735da27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:56:25.263697 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1455/2919] 10.222.0.30 () {58 vars in 958 bytes} [Tue Sep 6 04:56:25 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 04:56:25.275156 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025ca9e6-f605-4c59-b3ed-af4afbf2bd88 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:56:25.276863 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-025ca9e6-f605-4c59-b3ed-af4afbf2bd88 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:56:25.335876 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025ca9e6-f605-4c59-b3ed-af4afbf2bd88 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:56:25.339667 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe02a7c-1e70-4847-95b2-7ed0e6dd93ed service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd09b5ab675634a2eb4b03755743f7d0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b707733ce65c42edbc1fd95ad0a1c06c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:56:25.340310 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe02a7c-1e70-4847-95b2-7ed0e6dd93ed service cinder] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:56:25.340590 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe02a7c-1e70-4847-95b2-7ed0e6dd93ed service cinder] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:56:25.340883 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe02a7c-1e70-4847-95b2-7ed0e6dd93ed service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:56:25.350289 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe02a7c-1e70-4847-95b2-7ed0e6dd93ed service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:56:25.351718 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe02a7c-1e70-4847-95b2-7ed0e6dd93ed service cinder] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:56:25.360765 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1465/2920] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Sep 6 04:56:25 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 04:56:25.385188 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea88617e-4218-438e-85d6-c7db3735da27 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:56:25.389940 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:56:25.449261 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea88617e-4218-438e-85d6-c7db3735da27 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:56:25.454880 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e610018d-e796-4da5-b2d0-0f4ce8c6b4a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:56:25.455522 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e610018d-e796-4da5-b2d0-0f4ce8c6b4a1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:56:25.455805 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e610018d-e796-4da5-b2d0-0f4ce8c6b4a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:56:25.456084 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e610018d-e796-4da5-b2d0-0f4ce8c6b4a1 admin admin] PATH_INFO: `/v3/users/fbf2a82ddde7434c85a6ca84ea332026` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:56:25.457293 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e610018d-e796-4da5-b2d0-0f4ce8c6b4a1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fbf2a82ddde7434c85a6ca84ea332026)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:56:25.459237 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e610018d-e796-4da5-b2d0-0f4ce8c6b4a1 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:56:25.643276 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-e610018d-e796-4da5-b2d0-0f4ce8c6b4a1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fbf2a82ddde7434c85a6ca84ea332026'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:56:25.732634 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-e610018d-e796-4da5-b2d0-0f4ce8c6b4a1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fbf2a82ddde7434c85a6ca84ea332026'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:56:25.733510 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-e610018d-e796-4da5-b2d0-0f4ce8c6b4a1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fbf2a82ddde7434c85a6ca84ea332026'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:56:25.740816 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-e610018d-e796-4da5-b2d0-0f4ce8c6b4a1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fbf2a82ddde7434c85a6ca84ea332026'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:56:25.753962 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-e610018d-e796-4da5-b2d0-0f4ce8c6b4a1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fbf2a82ddde7434c85a6ca84ea332026'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:56:25.788754 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1456/2921] 10.222.0.30 () {58 vars in 1224 bytes} [Tue Sep 6 04:56:25 2022] DELETE /identity/v3/users/fbf2a82ddde7434c85a6ca84ea332026 => generated 0 bytes in 377 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 04:56:25.789001 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe12d4a-f092-4908-a674-ab0ab501a3fc None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:56:25.789001 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe12d4a-f092-4908-a674-ab0ab501a3fc None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:56:25.789001 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe12d4a-f092-4908-a674-ab0ab501a3fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:56:25.797883 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-efe12d4a-f092-4908-a674-ab0ab501a3fc None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:56:25.799862 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-efe12d4a-f092-4908-a674-ab0ab501a3fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:56:25.876291 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1466/2922] 10.222.0.30 () {58 vars in 958 bytes} [Tue Sep 6 04:56:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 04:56:25.891841 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e610018d-e796-4da5-b2d0-0f4ce8c6b4a1 admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:56:25.891841 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-e610018d-e796-4da5-b2d0-0f4ce8c6b4a1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:56:25.945981 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e610018d-e796-4da5-b2d0-0f4ce8c6b4a1 admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:56:25.949480 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476dc6b8-e38a-4a29-bdb9-acf029d266b0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90492440f6a6471e8392361c098428aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b707733ce65c42edbc1fd95ad0a1c06c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:56:25.950085 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476dc6b8-e38a-4a29-bdb9-acf029d266b0 service neutron] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:56:25.950362 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476dc6b8-e38a-4a29-bdb9-acf029d266b0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:56:25.950637 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476dc6b8-e38a-4a29-bdb9-acf029d266b0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:56:25.960299 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476dc6b8-e38a-4a29-bdb9-acf029d266b0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:56:25.961756 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476dc6b8-e38a-4a29-bdb9-acf029d266b0 service neutron] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:56:25.973678 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1457/2923] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Sep 6 04:56:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 04:56:26.468259 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe12d4a-f092-4908-a674-ab0ab501a3fc None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:56:26.468853 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-efe12d4a-f092-4908-a674-ab0ab501a3fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:56:26.481990 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe12d4a-f092-4908-a674-ab0ab501a3fc None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:56:26.487152 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd5bfde-a578-4816-bfdb-e5f0b91ef0f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:56:26.487783 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd5bfde-a578-4816-bfdb-e5f0b91ef0f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:56:26.488058 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd5bfde-a578-4816-bfdb-e5f0b91ef0f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:56:26.488329 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd5bfde-a578-4816-bfdb-e5f0b91ef0f5 admin admin] PATH_INFO: `/v3/projects/f23ca2e000164b9bada448dc01321bf8` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:56:26.490031 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd5bfde-a578-4816-bfdb-e5f0b91ef0f5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f23ca2e000164b9bada448dc01321bf8)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:56:26.492208 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd5bfde-a578-4816-bfdb-e5f0b91ef0f5 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:56:26.595851 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-8cd5bfde-a578-4816-bfdb-e5f0b91ef0f5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f23ca2e000164b9bada448dc01321bf8', 'request_id': 'req-8cd5bfde-a578-4816-bfdb-e5f0b91ef0f5'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:56:26.601254 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-8cd5bfde-a578-4816-bfdb-e5f0b91ef0f5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f23ca2e000164b9bada448dc01321bf8', 'request_id': 'req-8cd5bfde-a578-4816-bfdb-e5f0b91ef0f5'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:56:26.652903 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-8cd5bfde-a578-4816-bfdb-e5f0b91ef0f5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f23ca2e000164b9bada448dc01321bf8', 'request_id': 'req-8cd5bfde-a578-4816-bfdb-e5f0b91ef0f5'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:56:26.665244 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-8cd5bfde-a578-4816-bfdb-e5f0b91ef0f5 admin admin] The token cache is being invalidate because project f23ca2e000164b9bada448dc01321bf8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81698) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 04:56:26.665561 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-8cd5bfde-a578-4816-bfdb-e5f0b91ef0f5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:56:26.666187 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-8cd5bfde-a578-4816-bfdb-e5f0b91ef0f5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:56:26.667647 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1467/2924] 10.222.0.30 () {58 vars in 1233 bytes} [Tue Sep 6 04:56:26 2022] DELETE /identity/v3/projects/f23ca2e000164b9bada448dc01321bf8 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 04:56:26.673598 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa484096-0c8f-480d-bbb6-2149aa32a74c None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:56:26.674128 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa484096-0c8f-480d-bbb6-2149aa32a74c None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:56:26.674245 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa484096-0c8f-480d-bbb6-2149aa32a74c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:56:26.693373 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-fa484096-0c8f-480d-bbb6-2149aa32a74c None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:56:26.694558 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-fa484096-0c8f-480d-bbb6-2149aa32a74c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:56:26.713156 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:56:26.756092 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1458/2925] 10.222.0.30 () {58 vars in 958 bytes} [Tue Sep 6 04:56:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 04:56:26.764920 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f26133-c9ec-41fe-86f4-3f38ca34c07c None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:56:26.765202 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f26133-c9ec-41fe-86f4-3f38ca34c07c None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:56:26.765608 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f26133-c9ec-41fe-86f4-3f38ca34c07c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:56:26.789327 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-a7f26133-c9ec-41fe-86f4-3f38ca34c07c None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:56:26.790520 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-a7f26133-c9ec-41fe-86f4-3f38ca34c07c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:56:26.799009 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1468/2926] 10.222.0.30 () {58 vars in 958 bytes} [Tue Sep 6 04:56:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 04:56:26.803185 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa484096-0c8f-480d-bbb6-2149aa32a74c None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:56:26.813479 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa484096-0c8f-480d-bbb6-2149aa32a74c None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:56:26.818111 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a155ab9-6b03-4d04-aaa2-c0773aaf168a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:56:26.818716 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a155ab9-6b03-4d04-aaa2-c0773aaf168a admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:56:26.818998 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a155ab9-6b03-4d04-aaa2-c0773aaf168a admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:56:26.819272 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a155ab9-6b03-4d04-aaa2-c0773aaf168a admin admin] PATH_INFO: `/v3/domains` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:56:26.819924 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a155ab9-6b03-4d04-aaa2-c0773aaf168a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 04:56:26.820226 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a155ab9-6b03-4d04-aaa2-c0773aaf168a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:56:26.821858 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a155ab9-6b03-4d04-aaa2-c0773aaf168a admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:56:26.834205 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1459/2927] 10.222.0.30 () {58 vars in 1153 bytes} [Tue Sep 6 04:56:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 04:56:26.838281 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f26133-c9ec-41fe-86f4-3f38ca34c07c None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:56:26.848085 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f26133-c9ec-41fe-86f4-3f38ca34c07c None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:56:26.852653 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36168265-6747-44d7-94f0-95982c77dd80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:56:26.853246 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36168265-6747-44d7-94f0-95982c77dd80 admin admin] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:56:26.853553 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36168265-6747-44d7-94f0-95982c77dd80 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:56:26.853819 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36168265-6747-44d7-94f0-95982c77dd80 admin admin] PATH_INFO: `/v3/projects` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:56:26.854481 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36168265-6747-44d7-94f0-95982c77dd80 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:56:26.856391 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36168265-6747-44d7-94f0-95982c77dd80 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:56:26.860883 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:56:26.957057 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1469/2928] 10.222.0.30 () {60 vars in 1153 bytes} [Tue Sep 6 04:56:26 2022] POST /identity/v3/projects => generated 375 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 04:56:26.961514 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a155ab9-6b03-4d04-aaa2-c0773aaf168a admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:56:26.972655 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a155ab9-6b03-4d04-aaa2-c0773aaf168a admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:56:27.031826 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab37941-6abc-47de-89ab-4d4141a1b60c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:56:27.032797 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab37941-6abc-47de-89ab-4d4141a1b60c admin admin] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:56:27.033147 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab37941-6abc-47de-89ab-4d4141a1b60c admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:56:27.033485 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab37941-6abc-47de-89ab-4d4141a1b60c admin admin] PATH_INFO: `/v3/users` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:56:27.034125 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab37941-6abc-47de-89ab-4d4141a1b60c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:56:27.036587 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab37941-6abc-47de-89ab-4d4141a1b60c admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:56:27.150377 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1460/2929] 10.222.0.30 () {60 vars in 1144 bytes} [Tue Sep 6 04:56:26 2022] POST /identity/v3/users => generated 288 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 04:56:27.154530 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36168265-6747-44d7-94f0-95982c77dd80 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:56:27.165020 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36168265-6747-44d7-94f0-95982c77dd80 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:56:27.169673 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab5486f-73a7-458f-959c-9eb31e37d181 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:56:27.170270 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab5486f-73a7-458f-959c-9eb31e37d181 admin admin] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:56:27.171069 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab5486f-73a7-458f-959c-9eb31e37d181 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:56:27.171342 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab5486f-73a7-458f-959c-9eb31e37d181 admin admin] PATH_INFO: `/v3/roles` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:56:27.172178 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab5486f-73a7-458f-959c-9eb31e37d181 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:56:27.173797 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab5486f-73a7-458f-959c-9eb31e37d181 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:56:27.183611 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1470/2930] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Sep 6 04:56:27 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 04:56:27.188416 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab37941-6abc-47de-89ab-4d4141a1b60c admin admin] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:56:27.198330 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab37941-6abc-47de-89ab-4d4141a1b60c admin admin] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:56:27.203320 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3e5d4e-082e-42ef-8ac4-f4fe31b2ad41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:56:27.203918 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3e5d4e-082e-42ef-8ac4-f4fe31b2ad41 admin admin] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:56:27.204225 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3e5d4e-082e-42ef-8ac4-f4fe31b2ad41 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:56:27.204497 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3e5d4e-082e-42ef-8ac4-f4fe31b2ad41 admin admin] PATH_INFO: `/v3/roles` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:56:27.205131 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3e5d4e-082e-42ef-8ac4-f4fe31b2ad41 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:56:27.206760 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3e5d4e-082e-42ef-8ac4-f4fe31b2ad41 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:56:27.217825 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1461/2931] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Sep 6 04:56:27 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 04:56:27.221905 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab5486f-73a7-458f-959c-9eb31e37d181 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:56:27.231661 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab5486f-73a7-458f-959c-9eb31e37d181 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:56:27.236663 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89bc0b6f-0219-489f-b676-674c5cf1a440 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:56:27.237320 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bc0b6f-0219-489f-b676-674c5cf1a440 admin admin] REQUEST_METHOD: `PUT` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:56:27.237605 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bc0b6f-0219-489f-b676-674c5cf1a440 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:56:27.237875 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bc0b6f-0219-489f-b676-674c5cf1a440 admin admin] PATH_INFO: `/v3/projects/91d43687f40743dab7741801f3acc47d/users/4b6a7dffa116413bb0e61e05d9494273/roles/29607b2de39f40d68bb6ceef64b8a732` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:56:27.252197 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89bc0b6f-0219-489f-b676-674c5cf1a440 admin admin] RBAC: Authorizing `identity:create_grant(project_id=91d43687f40743dab7741801f3acc47d, user_id=4b6a7dffa116413bb0e61e05d9494273, role_id=29607b2de39f40d68bb6ceef64b8a732)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:56:27.254860 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89bc0b6f-0219-489f-b676-674c5cf1a440 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:56:27.345132 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1471/2932] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Sep 6 04:56:27 2022] PUT /identity/v3/projects/91d43687f40743dab7741801f3acc47d/users/4b6a7dffa116413bb0e61e05d9494273/roles/29607b2de39f40d68bb6ceef64b8a732 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 04:56:27.355541 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d88d2a-1e39-4879-8a1e-4c8fec130aff None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:56:27.355826 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d88d2a-1e39-4879-8a1e-4c8fec130aff None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:56:27.356096 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d88d2a-1e39-4879-8a1e-4c8fec130aff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:56:27.384475 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-43d88d2a-1e39-4879-8a1e-4c8fec130aff None None] MFA Rules not processed for user `4b6a7dffa116413bb0e61e05d9494273`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:56:27.385788 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-43d88d2a-1e39-4879-8a1e-4c8fec130aff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:56:27.449839 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1462/2933] 10.222.0.30 () {58 vars in 958 bytes} [Tue Sep 6 04:56:27 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 04:56:27.462430 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89bc0b6f-0219-489f-b676-674c5cf1a440 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:56:27.464095 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-89bc0b6f-0219-489f-b676-674c5cf1a440 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:56:27.519892 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89bc0b6f-0219-489f-b676-674c5cf1a440 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:56:27.523412 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3936d431-2a48-4be1-a633-ed61691ecd7a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd09b5ab675634a2eb4b03755743f7d0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b707733ce65c42edbc1fd95ad0a1c06c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:56:27.524008 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3936d431-2a48-4be1-a633-ed61691ecd7a service cinder] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:56:27.524285 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3936d431-2a48-4be1-a633-ed61691ecd7a service cinder] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:56:27.524552 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3936d431-2a48-4be1-a633-ed61691ecd7a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:56:27.533497 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3936d431-2a48-4be1-a633-ed61691ecd7a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:56:27.535180 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3936d431-2a48-4be1-a633-ed61691ecd7a service cinder] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:56:27.543798 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1472/2934] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Sep 6 04:56:27 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 04:56:29.566368 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d2d69c-e1cf-4374-9979-d37a2c3c78f0 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:56:29.566897 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d2d69c-e1cf-4374-9979-d37a2c3c78f0 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:56:29.567134 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d2d69c-e1cf-4374-9979-d37a2c3c78f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:56:29.586437 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-89d2d69c-e1cf-4374-9979-d37a2c3c78f0 None None] MFA Rules not processed for user `4b6a7dffa116413bb0e61e05d9494273`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:56:29.587669 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-89d2d69c-e1cf-4374-9979-d37a2c3c78f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:56:29.595114 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1463/2935] 10.222.0.30 () {58 vars in 958 bytes} [Tue Sep 6 04:56:29 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 04:56:29.640133 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3936d431-2a48-4be1-a633-ed61691ecd7a service cinder] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:56:29.641537 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-3936d431-2a48-4be1-a633-ed61691ecd7a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:56:29.698214 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3936d431-2a48-4be1-a633-ed61691ecd7a service cinder] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:56:29.701766 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f11095b-1d5a-49dd-98dd-21cbe1ed976f service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae5c31fb1d0342c0937bbc83a415113d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b707733ce65c42edbc1fd95ad0a1c06c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:56:29.702401 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f11095b-1d5a-49dd-98dd-21cbe1ed976f service glance] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:56:29.702674 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f11095b-1d5a-49dd-98dd-21cbe1ed976f service glance] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:56:29.702942 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f11095b-1d5a-49dd-98dd-21cbe1ed976f service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:56:29.704641 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-8f11095b-1d5a-49dd-98dd-21cbe1ed976f service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:56:29.771028 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:56:29.775262 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f11095b-1d5a-49dd-98dd-21cbe1ed976f service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:56:29.776695 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f11095b-1d5a-49dd-98dd-21cbe1ed976f service glance] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:56:29.788797 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1473/2936] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Sep 6 04:56:29 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 04:56:30.326072 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d2d69c-e1cf-4374-9979-d37a2c3c78f0 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:56:30.328204 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-89d2d69c-e1cf-4374-9979-d37a2c3c78f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:56:30.339080 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f11095b-1d5a-49dd-98dd-21cbe1ed976f service glance] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:56:30.340675 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-8f11095b-1d5a-49dd-98dd-21cbe1ed976f service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:56:30.343609 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:56:30.402647 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f11095b-1d5a-49dd-98dd-21cbe1ed976f service glance] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:56:30.407985 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6160b5dd-56a6-4def-a13f-cc22840d5b0c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90492440f6a6471e8392361c098428aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b707733ce65c42edbc1fd95ad0a1c06c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:56:30.408664 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6160b5dd-56a6-4def-a13f-cc22840d5b0c service neutron] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:56:30.408944 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6160b5dd-56a6-4def-a13f-cc22840d5b0c service neutron] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:56:30.409253 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6160b5dd-56a6-4def-a13f-cc22840d5b0c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:56:30.410614 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d2d69c-e1cf-4374-9979-d37a2c3c78f0 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:56:30.411503 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-6160b5dd-56a6-4def-a13f-cc22840d5b0c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:56:30.415734 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1de268-6165-4809-9906-9074caf54cb4 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d4f2e4dbd41441d84f03ddb30191e7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b707733ce65c42edbc1fd95ad0a1c06c', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:56:30.416336 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1de268-6165-4809-9906-9074caf54cb4 service placement] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:56:30.416613 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1de268-6165-4809-9906-9074caf54cb4 service placement] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:56:30.416880 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1de268-6165-4809-9906-9074caf54cb4 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:56:30.418749 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-5a1de268-6165-4809-9906-9074caf54cb4 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:56:30.425749 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6160b5dd-56a6-4def-a13f-cc22840d5b0c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:56:30.427316 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6160b5dd-56a6-4def-a13f-cc22840d5b0c service neutron] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:56:30.430518 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a1de268-6165-4809-9906-9074caf54cb4 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:56:30.432113 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a1de268-6165-4809-9906-9074caf54cb4 service placement] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:56:30.439569 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1474/2937] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Sep 6 04:56:30 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 06 04:56:30.446895 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1464/2938] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Sep 6 04:56:30 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 04:56:34.507003 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6160b5dd-56a6-4def-a13f-cc22840d5b0c service neutron] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:56:34.515792 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6160b5dd-56a6-4def-a13f-cc22840d5b0c service neutron] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:56:34.519443 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd87f47-131a-41ae-9b2c-b1a39e76f166 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90492440f6a6471e8392361c098428aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b707733ce65c42edbc1fd95ad0a1c06c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:56:34.520035 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd87f47-131a-41ae-9b2c-b1a39e76f166 service neutron] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:56:34.520313 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd87f47-131a-41ae-9b2c-b1a39e76f166 service neutron] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:56:34.520583 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd87f47-131a-41ae-9b2c-b1a39e76f166 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:56:34.522337 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-3fd87f47-131a-41ae-9b2c-b1a39e76f166 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:56:34.531906 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd87f47-131a-41ae-9b2c-b1a39e76f166 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:56:34.533359 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd87f47-131a-41ae-9b2c-b1a39e76f166 service neutron] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:56:34.541726 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1475/2939] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Sep 6 04:56:34 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 04:56:40.623948 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=26 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:56:42.651703 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1de268-6165-4809-9906-9074caf54cb4 service placement] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:56:42.653258 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-5a1de268-6165-4809-9906-9074caf54cb4 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:56:42.668340 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1de268-6165-4809-9906-9074caf54cb4 service placement] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:56:42.674606 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25cb7ab-b1b5-425c-9a65-e983f47f54af service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9db138b77afa4b068244b8ef715404fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b707733ce65c42edbc1fd95ad0a1c06c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:56:42.675241 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25cb7ab-b1b5-425c-9a65-e983f47f54af service nova] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:56:42.675519 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25cb7ab-b1b5-425c-9a65-e983f47f54af service nova] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:56:42.675789 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25cb7ab-b1b5-425c-9a65-e983f47f54af service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:56:42.677522 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-c25cb7ab-b1b5-425c-9a65-e983f47f54af service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:56:42.690373 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c25cb7ab-b1b5-425c-9a65-e983f47f54af service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:56:42.691945 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c25cb7ab-b1b5-425c-9a65-e983f47f54af service nova] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:56:42.702838 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1465/2940] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Sep 6 04:56:42 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 04:56:44.537998 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=35 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:56:49.957225 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6f4ce9-2f15-4e45-9a53-633f93a3fce6 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:56:49.957853 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6f4ce9-2f15-4e45-9a53-633f93a3fce6 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:56:49.957853 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6f4ce9-2f15-4e45-9a53-633f93a3fce6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:56:50.017345 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-3b6f4ce9-2f15-4e45-9a53-633f93a3fce6 None None] MFA Rules not processed for user `da5fed17101a4935bcc38b72caf9d263`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:56:50.018546 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-3b6f4ce9-2f15-4e45-9a53-633f93a3fce6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:56:50.080946 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1476/2941] 10.222.0.30 () {58 vars in 1009 bytes} [Tue Sep 6 04:56:49 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 04:56:50.219545 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25cb7ab-b1b5-425c-9a65-e983f47f54af service nova] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:56:50.220966 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-c25cb7ab-b1b5-425c-9a65-e983f47f54af service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:56:50.309723 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25cb7ab-b1b5-425c-9a65-e983f47f54af service nova] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:56:50.315653 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a788905-e852-4bc5-ac66-3aeaf1aa8505 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a676ec7f92a144edbeca8e6029ba9ee6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b707733ce65c42edbc1fd95ad0a1c06c', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:56:50.316258 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a788905-e852-4bc5-ac66-3aeaf1aa8505 service swift] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:56:50.316541 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a788905-e852-4bc5-ac66-3aeaf1aa8505 service swift] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:56:50.316819 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a788905-e852-4bc5-ac66-3aeaf1aa8505 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:56:50.324824 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a788905-e852-4bc5-ac66-3aeaf1aa8505 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:56:50.326432 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a788905-e852-4bc5-ac66-3aeaf1aa8505 service swift] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:56:50.334381 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1466/2942] 10.222.0.30 () {60 vars in 1384 bytes} [Tue Sep 6 04:56:50 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 06 04:56:51.162415 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd6f623-898a-4113-a615-14d80d1b2b01 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:56:51.162886 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd6f623-898a-4113-a615-14d80d1b2b01 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:56:51.162993 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd6f623-898a-4113-a615-14d80d1b2b01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:56:51.183291 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-dfd6f623-898a-4113-a615-14d80d1b2b01 None None] MFA Rules not processed for user `da5fed17101a4935bcc38b72caf9d263`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:56:51.184498 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-dfd6f623-898a-4113-a615-14d80d1b2b01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:56:51.192303 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1477/2943] 10.222.0.30 () {58 vars in 967 bytes} [Tue Sep 6 04:56:51 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 04:56:51.204437 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a788905-e852-4bc5-ac66-3aeaf1aa8505 service swift] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:56:51.215565 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a788905-e852-4bc5-ac66-3aeaf1aa8505 service swift] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:56:51.221356 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b0f572-dcc8-41ca-92d1-271bbb12e706 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a676ec7f92a144edbeca8e6029ba9ee6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b707733ce65c42edbc1fd95ad0a1c06c', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:56:51.221977 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b0f572-dcc8-41ca-92d1-271bbb12e706 service swift] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:56:51.222228 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b0f572-dcc8-41ca-92d1-271bbb12e706 service swift] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:56:51.222501 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b0f572-dcc8-41ca-92d1-271bbb12e706 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:56:51.230461 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b0f572-dcc8-41ca-92d1-271bbb12e706 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:56:51.232742 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b0f572-dcc8-41ca-92d1-271bbb12e706 service swift] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:56:51.240684 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1467/2944] 10.222.0.30 () {60 vars in 1384 bytes} [Tue Sep 6 04:56:51 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 06 04:56:51.859997 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d904dd8-3dd9-46ce-923d-4b38b2e3e0ca None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:56:51.860279 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d904dd8-3dd9-46ce-923d-4b38b2e3e0ca None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:56:51.860548 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d904dd8-3dd9-46ce-923d-4b38b2e3e0ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:56:51.881100 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-1d904dd8-3dd9-46ce-923d-4b38b2e3e0ca None None] MFA Rules not processed for user `da5fed17101a4935bcc38b72caf9d263`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:56:51.882306 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-1d904dd8-3dd9-46ce-923d-4b38b2e3e0ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:56:51.889221 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1478/2945] 10.222.0.30 () {58 vars in 1009 bytes} [Tue Sep 6 04:56:51 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 04:56:51.901141 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b0f572-dcc8-41ca-92d1-271bbb12e706 service swift] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:56:51.912608 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b0f572-dcc8-41ca-92d1-271bbb12e706 service swift] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:56:51.917685 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4810f128-3187-4305-8db3-4fdd6f2e5913 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a676ec7f92a144edbeca8e6029ba9ee6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b707733ce65c42edbc1fd95ad0a1c06c', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:56:51.918286 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4810f128-3187-4305-8db3-4fdd6f2e5913 service swift] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:56:51.918567 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4810f128-3187-4305-8db3-4fdd6f2e5913 service swift] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:56:51.918840 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4810f128-3187-4305-8db3-4fdd6f2e5913 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:56:51.927541 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4810f128-3187-4305-8db3-4fdd6f2e5913 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:56:51.929110 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4810f128-3187-4305-8db3-4fdd6f2e5913 service swift] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:56:51.936998 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1468/2946] 10.222.0.30 () {60 vars in 1384 bytes} [Tue Sep 6 04:56:51 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 06 04:56:52.488654 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d57e04-68d6-48e3-80f6-f72182d1b033 None None] REQUEST_METHOD: `POST` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:56:52.489141 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d57e04-68d6-48e3-80f6-f72182d1b033 None None] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:56:52.489248 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d57e04-68d6-48e3-80f6-f72182d1b033 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:56:52.508874 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.auth.core [None req-93d57e04-68d6-48e3-80f6-f72182d1b033 None None] MFA Rules not processed for user `da5fed17101a4935bcc38b72caf9d263`. Rule list: `[]` (Enabled: `True`). {{(pid=81698) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:56:52.510099 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-93d57e04-68d6-48e3-80f6-f72182d1b033 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:56:52.517061 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1479/2947] 10.222.0.30 () {58 vars in 967 bytes} [Tue Sep 6 04:56:52 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 04:56:52.529229 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4810f128-3187-4305-8db3-4fdd6f2e5913 service swift] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:56:52.541109 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4810f128-3187-4305-8db3-4fdd6f2e5913 service swift] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:56:52.546115 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b506c73c-3e13-490c-a8c0-8336a6294b48 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a676ec7f92a144edbeca8e6029ba9ee6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b707733ce65c42edbc1fd95ad0a1c06c', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:56:52.546709 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b506c73c-3e13-490c-a8c0-8336a6294b48 service swift] REQUEST_METHOD: `GET` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:56:52.546992 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b506c73c-3e13-490c-a8c0-8336a6294b48 service swift] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:56:52.547262 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b506c73c-3e13-490c-a8c0-8336a6294b48 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:56:52.555904 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b506c73c-3e13-490c-a8c0-8336a6294b48 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:56:52.557507 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b506c73c-3e13-490c-a8c0-8336a6294b48 service swift] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:56:52.566306 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1469/2948] 10.222.0.30 () {60 vars in 1384 bytes} [Tue Sep 6 04:56:52 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 06 04:56:55.207664 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d57e04-68d6-48e3-80f6-f72182d1b033 None None] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:56:55.269081 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d57e04-68d6-48e3-80f6-f72182d1b033 None None] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:56:55.273624 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db402020-bc38-47e7-8857-d739e271f921 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:56:55.274253 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db402020-bc38-47e7-8857-d739e271f921 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:56:55.274528 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db402020-bc38-47e7-8857-d739e271f921 admin admin] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:56:55.274798 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db402020-bc38-47e7-8857-d739e271f921 admin admin] PATH_INFO: `/v3/users/4b6a7dffa116413bb0e61e05d9494273` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:56:55.275975 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db402020-bc38-47e7-8857-d739e271f921 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4b6a7dffa116413bb0e61e05d9494273)` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:56:55.277964 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db402020-bc38-47e7-8857-d739e271f921 admin admin] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:56:55.473692 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:56:55.475166 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-db402020-bc38-47e7-8857-d739e271f921 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4b6a7dffa116413bb0e61e05d9494273'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:56:55.546567 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-db402020-bc38-47e7-8857-d739e271f921 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4b6a7dffa116413bb0e61e05d9494273'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:56:55.547209 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-db402020-bc38-47e7-8857-d739e271f921 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4b6a7dffa116413bb0e61e05d9494273'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:56:55.559804 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-db402020-bc38-47e7-8857-d739e271f921 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4b6a7dffa116413bb0e61e05d9494273'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:56:55.567109 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.notifications [None req-db402020-bc38-47e7-8857-d739e271f921 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4b6a7dffa116413bb0e61e05d9494273'} {{(pid=81698) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:56:55.573215 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1480/2949] 10.222.0.30 () {58 vars in 1224 bytes} [Tue Sep 6 04:56:55 2022] DELETE /identity/v3/users/4b6a7dffa116413bb0e61e05d9494273 => generated 0 bytes in 366 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 04:56:55.577810 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5fe42b-8a1d-4b73-b335-ef8d09c1adc2 None None] REQUEST_METHOD: `POST` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:56:55.578309 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5fe42b-8a1d-4b73-b335-ef8d09c1adc2 None None] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:56:55.578437 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5fe42b-8a1d-4b73-b335-ef8d09c1adc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:56:55.598059 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.auth.core [None req-3e5fe42b-8a1d-4b73-b335-ef8d09c1adc2 None None] MFA Rules not processed for user `b7c5062264774b85bf945fb9ac7602db`. Rule list: `[]` (Enabled: `True`). {{(pid=81697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 04:56:55.599268 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-3e5fe42b-8a1d-4b73-b335-ef8d09c1adc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:56:55.659033 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1470/2950] 10.222.0.30 () {58 vars in 958 bytes} [Tue Sep 6 04:56:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 04:56:55.668395 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db402020-bc38-47e7-8857-d739e271f921 admin admin] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:56:55.669835 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-db402020-bc38-47e7-8857-d739e271f921 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:56:55.725851 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db402020-bc38-47e7-8857-d739e271f921 admin admin] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:56:55.729369 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e587de-c873-412c-be50-b8ce6cb4b341 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90492440f6a6471e8392361c098428aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b707733ce65c42edbc1fd95ad0a1c06c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:56:55.730073 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e587de-c873-412c-be50-b8ce6cb4b341 service neutron] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:56:55.730354 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e587de-c873-412c-be50-b8ce6cb4b341 service neutron] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:56:55.730623 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e587de-c873-412c-be50-b8ce6cb4b341 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:56:55.740189 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e587de-c873-412c-be50-b8ce6cb4b341 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:56:55.741671 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e587de-c873-412c-be50-b8ce6cb4b341 service neutron] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:56:55.751120 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1481/2951] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Sep 6 04:56:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 04:56:55.995177 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5fe42b-8a1d-4b73-b335-ef8d09c1adc2 None None] Authenticating user token {{(pid=81697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:56:55.996729 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.fernet_utils [None req-3e5fe42b-8a1d-4b73-b335-ef8d09c1adc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:56:56.009207 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5fe42b-8a1d-4b73-b335-ef8d09c1adc2 None None] Validating token access rules against request {{(pid=81697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:56:56.013831 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dab4d55-3d85-4454-b5a8-9bff457512f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7c5062264774b85bf945fb9ac7602db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b3067b612904279b76189015d91b056', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:56:56.014474 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dab4d55-3d85-4454-b5a8-9bff457512f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:56:56.014757 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dab4d55-3d85-4454-b5a8-9bff457512f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:56:56.015032 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dab4d55-3d85-4454-b5a8-9bff457512f8 admin admin] PATH_INFO: `/v3/projects/91d43687f40743dab7741801f3acc47d` {{(pid=81697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:56:56.016274 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dab4d55-3d85-4454-b5a8-9bff457512f8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=91d43687f40743dab7741801f3acc47d)` {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:56:56.019098 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dab4d55-3d85-4454-b5a8-9bff457512f8 admin admin] RBAC: Authorization granted {{(pid=81697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:56:56.031062 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=100 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:56:56.115746 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-7dab4d55-3d85-4454-b5a8-9bff457512f8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '91d43687f40743dab7741801f3acc47d', 'request_id': 'req-7dab4d55-3d85-4454-b5a8-9bff457512f8'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:56:56.172291 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-7dab4d55-3d85-4454-b5a8-9bff457512f8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '91d43687f40743dab7741801f3acc47d', 'request_id': 'req-7dab4d55-3d85-4454-b5a8-9bff457512f8'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:56:56.178829 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-7dab4d55-3d85-4454-b5a8-9bff457512f8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '91d43687f40743dab7741801f3acc47d', 'request_id': 'req-7dab4d55-3d85-4454-b5a8-9bff457512f8'} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:56:56.189034 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-7dab4d55-3d85-4454-b5a8-9bff457512f8 admin admin] The token cache is being invalidate because project 91d43687f40743dab7741801f3acc47d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81697) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 04:56:56.189350 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-7dab4d55-3d85-4454-b5a8-9bff457512f8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:56:56.190012 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG keystone.notifications [None req-7dab4d55-3d85-4454-b5a8-9bff457512f8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 04:56:56.191901 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: [pid: 81697|app: 0|req: 1471/2952] 10.222.0.30 () {58 vars in 1233 bytes} [Tue Sep 6 04:56:55 2022] DELETE /identity/v3/projects/91d43687f40743dab7741801f3acc47d => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 04:57:05.746920 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=39,keystone:INSERT=1,keystone:DELETE=3 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:57:06.186058 dv-cinder-856002-1-iscsi devstack@keystone.service[81697]: DEBUG dbcounter [-] [81697] Writing DB stats keystone:SELECT=19,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 04:59:43.294791 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e587de-c873-412c-be50-b8ce6cb4b341 service neutron] Authenticating user token {{(pid=81698) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 04:59:43.296180 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-b3e587de-c873-412c-be50-b8ce6cb4b341 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:59:43.365497 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e587de-c873-412c-be50-b8ce6cb4b341 service neutron] Validating token access rules against request {{(pid=81698) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 04:59:43.426549 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e3c7e8-2b2e-4c22-b86b-7bd588c42367 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d4f2e4dbd41441d84f03ddb30191e7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b707733ce65c42edbc1fd95ad0a1c06c', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81698) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 04:59:43.429953 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e3c7e8-2b2e-4c22-b86b-7bd588c42367 service placement] REQUEST_METHOD: `GET` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 04:59:43.429953 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e3c7e8-2b2e-4c22-b86b-7bd588c42367 service placement] SCRIPT_NAME: `/identity` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 04:59:43.429953 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e3c7e8-2b2e-4c22-b86b-7bd588c42367 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81698) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 04:59:43.430350 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.fernet_utils [None req-55e3c7e8-2b2e-4c22-b86b-7bd588c42367 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81698) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 04:59:43.441461 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e3c7e8-2b2e-4c22-b86b-7bd588c42367 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 04:59:43.443042 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e3c7e8-2b2e-4c22-b86b-7bd588c42367 service placement] RBAC: Authorization granted {{(pid=81698) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 04:59:43.453962 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: [pid: 81698|app: 0|req: 1482/2953] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Sep 6 04:59:43 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 04:59:53.449699 dv-cinder-856002-1-iscsi devstack@keystone.service[81698]: DEBUG dbcounter [-] [81698] Writing DB stats keystone:SELECT=58 {{(pid=81698) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}